The primary goals of this assignment are:
- Create a UE4 Plugin from scratch.
- Create a UE4 object importer.
- Get experience looking through other plugin code and engine source as a strategy for working with a large codebase.
For this assignment, you'll be creating an importer plugin to load a UVolumeTexture object from a file. The importer will load files in a subset of the NRRD (Nearly Raw Raster Data) format into an RGBA16F texture. The NRRD format consists of a simple text header describing the file contents, coupled with binary image data, either embedded in the same file (typically using the .nrrd extension) or in a separate file in the same directory (typically using the .nhdr extension for the header, and .raw or .dat for the data). The primary file extension can be either .nrrd or .nhdr. Though there is a convention that detached header files use nhdr, the format does not distinguish between the two extensions. Similarly, there is a convention to use .raw or .dat for the binary data, but it can legally use any extension or none at all.
There are a ton of possible header lines in the NRRD format, most of which are meaningless when just loading a texture. These are the lines you should parse from the present.nhdr file
NRRD0004 type: uint16 dimension: 3 sizes: 492 492 442 data file: present_492x492x442_uint16.raw
file identifier, must be first line. element data type with 'kinds', identifies this as 3D volume data data dimensions: read from file, do not hard-code name of data file in same directory as .nhdr file
The actual present.nhdr file provided includes more header lines than are listed here. You should ignore lines your importer does not support. Your importer also only needs to support one-channel volumes with uint16 components and the data in a separate file (like this one). Check the NRRD documentation to make sure you can handle any legal keyword variations. For example, these are all equivalent: "uint16", "ushort", "unsigned short", "unsigned short int", and "uint16_t" as equivalent. So is "data file", or "datafile".
The RGBA16F texture format holds a 16-bit floating point value for each of four color channels. UE4 includes a class, FFloat16, to represent and convert between normal 32-bit single precision floats and this GPU-specific floating point type. It also includes another class, FFloat16Color, for RGBA colors using this type. To convert the uint16 into a floating point color value, you should scan the entire data file to find the minimum and maximum value present and re-scale the values to map the minimum to a floating point value of 0 and the maximum to a floating point value of 1 (e.g. using (val-min)/(max-min)). When storing data with fewer channels into an RGBA color, you'll generally replicate the same value into multiple channels or set some reasonable default for some of the extra channels. In our case, we'll copy each input value into all four channels.
This assignment provides only rough pointers to where to find the UE4 examples and class declarations you will need. Part of the assignment is for you to get more comfortable searching through the source to find things like this.
Create a project
- Create a Blank C++ project with no starter content called
- As usual, put the project at the top level of your git repository
- Unzip the testing data files, and keep both the nhdr and raw files in the same directory. You do not need to copy these into your repository.
- I downloaded this file from Open Scientific Visualization Datasets, which also has some additional files you can try.
Create the plugin
- This time you will start from a Blank code plugin.
- Add an asset load factory class to your plugin.
- Classes to load assets are derived from UFactory.
- To load from a file, you should override the UFactory::FactoryCreateFile function. Find examples in the engine and engine plugins.
- To avoid undefined symbol errors, add the module for UFactory to your plugin's Build.cs. Look at the UFactory *_API macro or follow the directory tree to find its Build.cs file to find the module name.
- Use Formats.Add in your class constructor to tell UE4 that you handle files with the nhdr and nrrd file extensions. Note that the code that implements Formats.Add cannot handle extra spaces before the extension, or before or after the ";" that separates the extension from description.
- At this point, if your importer creates and returns an empty UVolumeTexture object, you should be able to drag an nhdr file into the Content window and get an empty volume texture object to confirm that your format registration code is correct.
- Delete the old imported asset before trying to import again. Assuming you have not actually used it for anything (added it to any materials or as a property on another object), it is safe to "Force Delete" when removing the object.
- Load and parse the nhdr header file
- I recommend reading the file by lines, splitting each line at the ":", and using simple string comparison or conversion to parse.
- Look at FFileHelper for file loading functions.
- Look at FString and FCString for simple string matching and string-to-integer conversion functions.
- If things go wrong, use UE_LOG to send a message to the Editor Output window and return nullptr.
- Load the data
- Use FScopedSlowTask to display a progress bar and update it during loading
- FPaths has some helpful functions for splitting a file name into the path, base, and extension
- Load the data into the UVolumeTexture
- Set UTexture::SRGB to 0, since this is not color data
- Set UTexture::MipGenSettings to TextureMipGenSettings::TMGS_NoMipmaps to avoid generating multiple resolution versions of the volume data (which may fail if the dimension are not a power of two).
- Texture.Source.Init will initialize the size, format, and data
- Call UTexture::UpdateResource to update the UVolumeTexture object with the loaded data
Graduate students must also support nrrd files with embedded data. These files do not include a "data file" header line. There will be a blank line at the end of the header, with data starting immediately after the '\n' at the end of this blank line.
Extra credit is only available if you submit by the original due date. For up to five points each, add one or more of the following features.
- Robustly report loading errors in NRRD file for any of the header lines you do use (sizes you don't support, mismatches between sizes and dimensions, etc.). You still do not need to report errors for header lines you do not support at all.
- Support for 2D textures (with dimensions = 2 and only two numbers in the sizes line).
- Support for uint8 and float data. Float data does not need to be scaled to the data min and max.
- Support for the "byte skip" header. This is handy to directly use other uncompressed data files with embedded header information as the raw file. The original source for the present dataset is such a file (with six bytes giving the volume size, followed by the raw data).
- Support for (all of) 2, 3, and 4 channel data.
Create and submit test files for any features you add. The easiest way to create test files this is to write a simple standalone program to write them. You do not need to submit any data generation programs, just the resulting test data files.
The last of the extra credit options (multi-channel data) will have an extra dimension, with a different "kind" given for the first dimension: 2-vector; 3-color, 3-vector, or RGB-color; and 4-color, 4-vector or RGBA-color. You will need to add parsing for the "kind" header line to distinguish 2D color images from 3D single-channel volumes. For example, here are 1-channel and 3-channel headers for a 2D image:
NRRD0004 type: uint8 dimension: 2 sizes: 256 256 kinds: domain domain data file: grey-image.raw
NRRD0004 type: uint8 dimension: 3 sizes: 3 256 256 kinds: RGB-color domain domain data file: color-image.raw
Assume two channel images are intensity/opacity, so should have the first channel copied into all three of RGB, and the second channel copied into opacity. Three-channel data should put the three data channels into RGB and set A to 1.
For full credit, you must commit multiple times during your development.
assn5.txt to the top directory. Tell us what works and what doesn't, and anything else you think we should know for grading.
For the base assignment, you can include two screen shots of the UE4 editor pane for your VolumeTexture object. One should be in "Depth Slices" view mode, and the other in "Trace Into Volume" view mode.
Make sure your
assn5.txt file tells us if you attempted the extra credit, and if so, which parts, and where to find the test file(s), and screen shots showing your results. Do commit your test nrrd/nhdr and raw files so we can try them.
Record video of you importing your file(s), then showing the loaded object in the UE4 Editor image panel. This time, since we want to see you interacting with the editor, you should use an external video recording tool. For any volume object, show it in both "Depth Slices" and "Trace Into Volume" view mode. Upload the video to google drive, youtube, or similar, and include a link to it in your assn5.txt file. For extra credit, include a separate video for each extra credit element you attempted. Do not commit copies of your video to your repository.
Push to your repository, and tag your final commit with an
assn5 tag. Make sure the images and video links are included in the commit that you tag for us to grade.