Combined User Install Guide
- Contents
- Launching MIST
- Installation Validation Test
- MIST Graphical User Interface
- Input Parameters
- Output Parameters
- Subgrid Parameters
- Advanced Parameters
- Help
- MIST Status Frame
- MIST Plugin Installation
- Update Site Installation
- Manual Installation
- FFTW Installation
- FFTW Installation on Unix (64-bit Mac OSX/Linux)
- FFTW Installation using MacPorts
- CUDA Installation
- CUDA Toolkit v6.5 installation on Windows/Linux/MacOSX
- Contents
- My image filename pattern does not work
- Sequential
- Row Column
- Timeslices
- MIST encountered an Out of Memory problem
- General Memory Management
- Java Mode
- FFTW Mode
- CUDA Mode
- What image types can MIST handle
- My stitching results look incorrect
- Incorrect Input Parameters
- Unexpected Image Overlap Variability
- Bad Stage Repeatability Estimate
- Noisy Input Images
- What if the output stitched image is too big to be saved
- I installed the CUDA toolkit, but I am getting a CUDA Exception
- The FFTW version does not work
- Windows
- OSX
- Linux
- My stitching performance is not good
- Image Locality
- Memory Capacity
- CPU Saturation
- I have a GPU in my machine but it is not showing up
- How to stitch from metadata
- Stitching Time-Slices
- Stitching Multiple Channels
- Stitching using a Macro Script
- Submitting issue or bug report
- Start Fiji.
- Click through the menus: Plugins >> Stitching >> MIST
- Click: MIST
In this section, we will walk through setting up running MIST on one of the 5x5 image tile datasets in order to validate that the plugin is installed correctly.
- Download test images: Cy5_ImageTiles.zip ~ 54 MB
- Extract Cy5_ImageTiles.zip into a directory
- Launch MIST plugin from Fiji
- Select Load Params and open '[extractionPath]/Small_Fluorescent_Test_Dataset/stitching-params.txt'
- To manually enter parameters (avoiding Load Params) enter the following values into the Input Tab
- Filename Pattern Type: 'Row-Column'
- Filename Pattern: 'img_r{rrr}_c{ccc}.tif'
- Starting Point: 'Upper Left'
- Grid Width: '5'
- Grid Height: '5'
- Modify Image Directory to '[extractionPath]/Small_Fluorescent_Test_Dataset/image-tiles'
- Select the Output tab and update Output Directory to define where the output files are to be saved
- Set Blending Mode: to 'Overlay'
- Update Display Stitched Image and Save Full Stitched Image to selected/enabled
- To check the parameters defining the image grid, select Preview (0% overlap) which should generate and display a zero percent overlap tiling of the input images
- Click Begin Stitching to launch
- Compare the newly generated stitched image to the reference stitched image shipped with the test dataset
- Open '[extractionPath]/Small_Fluorescent_Test_Dataset/stitched-image/img-stitched-0.tif' in Fiji.
- Check that the new stitched image has the same dimensions as the reference
- Perform a pixel-wise comparison between the stitched images 1. Fiji>>Process>>Image Calculator 2. The operation new stitched image "Difference" reference stitched image should produce an image containing all zeros when using the correct parameters
The MIST plugin consists of five tabs. Below, we outline each tab and discuss all of the parameters. All options labeled with an * are required in order to start stitching. Each tab contains a ? button that, when clicked, generates a local copy of this documentation.
Save Params Button - Saves all parameters in MIST to a file.
Preview (0% overlap) Button - Opens the image mosaic into Fiji assuming 0% overlap.
Begin Stitching Button - Launches the stitching experiment.
Load Params Button - Loads parameters from a file. Note: Supports loading the statistics file that is generated after each experiment.
* Filename Pattern Type - Used to specify the type of filename pattern used in the acquired images. Possible options: Sequential and Row-Column.
Sequential:
Sequential pattern images have a single number representing their position within the image grid. Example: img_pos001_c01.tif = img_pos{ppp}_c01.tif {ppp} - Special text that represents position numbering between 0 and 999. Increase the number of p's to represent larger numbers.Row-Column:
Row-Column pattern images have a pair of numbers representing their position within the image grid. Example: img_r01_c01.tif = img_r{rr}_c{cc}.tif {rr} - Special text that represents row numbering between 0 and 99. Increase the number of r's to represent larger numbers. {cc} - Special text that represents column numbering between 0 and 99. Increase the number of c's to represent larger numbers.
* Starting Point - The starting point of the microscope scan. Possible options: Upper Left, Upper Right, Lower Left, and Lower Right. This specifies the origin for the grid of images.
* Direction - The direction and pattern of the microscope motion during acquisition. Possible options: Vertical Combing, Vertical Continuous, Horizontal Combing, and Horizontal Continuous.
* Grid Width - The number of images in a row (The number of columns, the width of the image grid)
* Grid Height - The number of images in a column. (The number of rows, the height of the image grid)
Timeslices - The number of timeslices to stitch. Leave this field blank to stitch all timeslices. To stitch timeslices you must add the spcial format text {ttt} to the File Pattern. This input supports a comma separated list and/or a range using a '-'. Example: "1-25,35,45" stitches timeslices 1 through 25, 35, and 45.
Discover Width/Height Button - Attempts to automatically discover the width and height of a row-column filename pattern type acquisition given a valid filename pattern and image directory.
* Filename Pattern - Used to load images from a directory. Changes based on the Filename Pattern Type. Important: the character "%" cannot be used as part of a filename pattern.
- If the Filename Pattern Type = Sequential, then the Filename Pattern expects sequential style numbering. We support all filename types that ImageJ supports. For a list of supported types, please refer to ImageJ User Guide
Example 1: Img_pos0001_c01.tif = Img_pos{pppp}_c01.tif Examples 2: Img_pos001_time001.tif = Img_pos{ppp}_time{ttt}.tif {ppp} - Special text that represents position numbering between 0 and 999. Increase the number of p's to represent larger numbers. {ttt} - Special text that represents timeslice number between 0 and 999. Each value of {ttt} will be stitched independently. Increase the number of t's to represent larger numbers. Specifying time slices is optional.
- If the Filename Pattern Type = Row-Column, then the Filename Pattern expects row-column style numbering.
Example 1: Img_r1_c1.tif = Img_r{r}_c{c}.tif Example 2: Img_row01_col01_time01.tif = Img_row{rr}_col{cc}_time{tt}.tif {rr} - Special text for row numbering between 0 and 99. Increase the number of r's to represent larger numbers. {cc} - Special text for column numbering between 0 and 99. Increase the number of c's to represent larger numbers. {ttt} - Special text that represents timeslice number between 0 and 999. Each value of {ttt} will be stitched independently. Increase the number of t's to represent larger numbers. Specifying time slices is optional.
* Image Directory - The directory where the source images are located. Select browse to open a directory browser.
Assemble from metadata - Generates the image mosaic using metadata from a previous run. Important: You must specify the global-positions file to use in the Global Positions File field. If your Filename Pattern has a time slice iterator "{tt}" then the global-positions file specified must also.
Global Positions File - The filepath to the global-positions file to use in assembling from metadata. Note: this must be the filepath of an individual global-positions file, not the directory containing a global-positions file. Note: If you Filename Pattern has a time slice iterator "{tt}" then the global-positions file specified must also.
Use Image Directory as Output Directory - Sets the output directory to be the same as the input directory.
Output Directory - Specifies the location to save images and run statistics.
Files that are saved:
- {namePrefix}stitched-{#}.tiff
- **{namePrefix}**statistics.txt
- global-positions-{#}.txt
- relative-positions-{#}.txt
- relative-positions-no-optimization-{#}.txt
- hillclimb-startings-positions-{#}.txt {namePrefix} - the prefix used in "Filename Prefix" {#} - The timeslice number (0 if no timeslices are used)
Filename Prefix - The prefix given to output files.
Blending mode - Selects the blending mode to be used for computing the mosaic image. Blending Types:
- Overlay - Choose only one pixel from overlapping pixels based on highest accuracy
- Linear - Smoothly alters the intensity of the overlapping area between images. Smoothness is determined by alpha. Value should be between (0, 5]
- Average - Computes the average intensity
Warning:: RGB images may cause out of memory issues for linear and average blending. When generating the mosaic of RBG images it is recommended to use overlay images.
Display Stitched Image - Opens the blended image mosaic into Fiji.
Save Full Stitched Image - Writes the blended image mosaic to the Output Directory.
Update Button - Updates the estimated stitched image file size.
Warning: Large image grids can cause out of memory errors. If the number of pixels in the output stitched image exceeds 2147483647 (231-1), then the stitched image must be exported using an alternative method. (imagePixelWidth*imagePixelHeight < 2147483647)
Use full grid - Updates the subgrid to use the full grid of images.
Start Col - The start column for the subgrid.
Start Row - The start row for the subgrid.
Extent Width- The width for the subgrid.
Extent Height - The height for the subgrid.
Suppress SubGrid Warning Dialog Box - suppress the warning dialog box that gets displayed when stitching a subgrid. The warning dialog is useful if you infrequently stitch subgrids to prevent inadvertently stitching a subgrid.
Stage Repeatability - Sets the stage repeatability variable when computing the global optimization. This value is used to represent the repeatability of the microscope stage movement. It is used for determining the search space of the hill climbing algorithm. This value is specified in pixels.
Horizontal overlap - Sets the horizontal overlap variable when computing the global optimization. This value is used to filter translations as good or bad. Good translations are used to identify the optimal starting position for the hill climbing algorithm. Setting this value can improve the accuracy of the image mosaic. This value is specified in percent. The value must be between 0 and 100.
Vertical overlap - Sets the vertical overlap variable when computing the global optimization. This value is used to filter translations as good or bad. Good translations are used to identify the optimal starting position for the hill climbing algorithm. Setting this value can improve the accuracy of the image mosaic. This value is specified in percent. The value must be between 0 and 100.
Overlap uncertainty - Sets the overlap uncertainty variable when computing the global optimization. This value is used to specify the range when filtering translations as good or bad. It is used in conjunction with the horizontal and vertical overlap variables. This value is specified in percent. The value must be between 0 and 100. This value should generally not exceed 10. (Default: 3)
Use BioFormats Image Reader? - Specifies whether to use the BioFormats image reader as opposed to the ImageJ image reader. The ImageJ reader supports most common image file types and is faster than BioFormats. If the ImageJ reader cannot read an image being stitched, try using BioFormats as it supports more image types.
Use Double Precision Math? - Specifies whether the FFTs and other stitching computation is done using 32bit floats of 64bit doubles. The default is 32bit, but if you are not getting a satisfactory stitching try using double precision.
Translation Refinement Method - Specifies the type of translation refinement that is performed. Once the translations between images have been computed, the stage model is built to constrain the translations to reasonable values. Within the search space defined by the stage model an optimization of the pairwise translations takes place. The method of translation optimization/refinement is selected here. The default is a single hill climbing starting at the computed/estimated translation. This is the fastest option. If this proves ineffective for your dataset, multipoint hill climbing is available where, in addition to performing a hill climbing starting at the computed/estimated translation, n starting points are randomly selected within the search bounds and allowed to converge to local maxima. The number of starting points is user selected (default of 16). This is more robust than the single hill climb, but more computationally intensive. The final option, Exhaustive, checks every possible translation within the stage model bounds. This is extremely slow.
Number of FFT Peaks - Specifies the number of peaks to check when computing the phase correlation image alignment method. We have determined that modifying this value can yield more accurate pre-optimization displacements. This value should not exceed 10. (Default: 2)
Log Level - Sets the log level for the user interface and the stitching execution. Possible Options:
- None - No logging
- Mandatory - Logs important information
- Helpful - Logs helpful information
- Info - Logs informative information
- Verbose - Logs all information
Note: The lower logging stack includes the logging levels above. Example: Helpful, includes Mandatory.
Debug Level - Sets the debug level for the user interface and the stitching execution. Possible Options:
- None - No debug logging
- Mandatory - Logs important debug information
- Helpful - Logs helpful debug information
- Info - Logs informative debug information
- Verbose - Logs all debug information
Note: The lower logging stack includes the logging levels above. Example: Helpful, includes Mandatory.
Stitching Program - Selects which type of program to execute.
- Auto - Determine which program to use. First checks if FFTW is available, if it is not then it falls back to Java.
- Java - Uses the pure java implementation. This version uses single precision values, but is not as accurate as FFTW and CUDA.
- FFTW - Uses FFTW libraries.
- CUDA - Uses NVIDIA CUDA GPUs.
CPU worker threads - Number of parallel workers to spawn (should not exceed number of CPU cores)
CPU worker threads - Number of parallel workers to spawn (should not exceed number of CPU cores)
FFTW Plan Type - The type of FFTW plan to generate
- Measure - Default, short creation time, usually ideal runtimes
- Patient - Long planning time, more robust runtimes
- Exhaustive - Very long planning time, most robust runtimes
Note: FFTW plans can be saved and reused to eliminate planning time.
Save Plan? - Saves the FFTW plan that is generated to file.
Load Plan? - Loads the FFTW plan from file.
FFTW Library File - Specifies the FFTW library file to load. Example Files:
- Windows: libfftw3.dll (this library is packaged with MIST in the "Fiji.app/lib/fftw" directory)
- Linux: libfftw3.so (default installation location "/usr/local/lib")
- Mac OSX: libfftw3.dylib (default installation location "/usr/local/lib")
Plan Location (or file) - The location to load FFTW plans. If a file is specified, the file will be used when loading the plan. This path is also used for saving plans.
CPU worker threads - Number of parallel workers to spawn (should not exceed number of CPU cores)
Execute Device Query - Queries for all GPUs installed on the system and prints the information to the log.
Refresh Device Table - Forces the table to re-query all GPUs and repopulate the table.
GPU Table - Displays all available GPUs found on the machine. To use one (or more) GPUs, click the check box in the Selected? column. Note: GPUs have limited memories, it is recommended to use GPUs with at least 1 GB of graphics memory depending on the size of the image grid.
Open Local Help Documentation - Opens this document without an Internet connection.
Progress - If timeslices are used, shows the current timeslice, total timeslices, and number of groups (if entered as a comma separated list in the input parameters).
Progress bar - Shows the % progress made for the current experiment.
Log Level - Sets the log level for the user interface and the stitching execution. Possible Options:
- None - No logging
- Mandatory - Logs important information
- Helpful - Logs helpful information
- Info - Logs informative information
- Verbose - Logs all information
Note: The lower logging stack includes the logging levels above. Example: Helpful, includes Mandatory.
Debug Level - Sets the debug level for the user interface and the stitching execution. Possible Options:
- None - No debug logging
- Mandatory - Logs important debug information
- Helpful - Logs helpful debug information
- Info - Logs informative debug information
- Verbose - Logs all debug information
Note: The lower logging stack includes the logging levels above. Example: Helpful, includes Mandatory.
Cancel Execution - Safely stops execution and releases used resources.
* - required in order to start stitching
Installing the plugin into Fiji is simplified using Fiji's update site manager. If the machine is not configured for the network then please refer to the manual installation. An example of how to install using Fiji's update site manager can be found HERE. Detailed instructions for installing MIST using the update site are below.
- Open Fiji
- Help >> Update Fiji (or "Update...")
- Manage update sites
- Activate MIST by selecting check box
- Close and Apply Changes
- Restart Fiji
- Plugin is installed in Plugin >> Stitching >> MIST
Or
- Open Fiji
- Help >> Update Fiji (or "Update...")
- Manage update sites
- Click "Add"
- Enter Name: "MIST" and URL: http://sites.imagej.net/NIST-ISG-MIST/
- Close and Apply Changes
- Restart Fiji
- Plugin is installed in Plugin >> Stitching >> MIST
- Download the latest version of the MIST plugin from MIST Plugin Download
- Extract the contents of the zip file into your Fiji.app folder
- Restart Fiji
- Plugin is installed in Plugins >> Stitching >> MIST
FFTW (Fastest Fourer Transform in the West, http://fftw.org ) is a C implementation out of MIT that computes the discrete Fourier transform. The plugin loads the library at run-time and uses the high performance FFTW functions in Java.
Requirements:
- Fiji (64-bit version)
Note: 64-bit Windows systems should be able to run the FFTW version without additional setup.
- Download fftw-3.3.4.tar.gz from http://www.fftw.org/download.html
- Extract source code to directory (terminal command: "tar -xzvf fftw-3.3.4.tar.gz")
- From a terminal locate source code and run the following commands to compile the 32bit and 64bit versions:
./configure --enable-shared make sudo make install ./configure --enable-float --enable-shared make sudo make install
(If configure fails, additional compilers and/or libraries may be needed)
- Launch the Fiji MIST plugin
- Specify the FFTW library file in the Advanced FFTW Options FFTW options user guide (default FFTW installation location is in /usr/local/lib)
- Linux library name example: libfftw3.so (or similar)
- MacOSX library name example: libfftw3.dylib (or similar)
- You should now be able to launch MIST using FFTW
- From the command-line:
sudo port install fftw-3 - Launch the Fiji MIST plugin
- Specify the FFTW library file in the Advanced FFTW Options FFTW options user guide (default FFTW installation location is in /usr/local/lib)
- Linux library name example: libfftw3.so (or similar)
- MacOSX library name example: libfftw3.dylib (or similar)
- You should now be able to launch MIST using FFTW
CUDA is the Compute Unified Device Architecture and is a language developed by NVIDIA. It is used to execute code on NVIDIA GPUs to obtain high performance.
Requirements:
- NVIDIA GPU installed on your system (compute capability 3.0 or better, equivalent to Kepler architecture or newer)
- CUDA toolkit version 6.5
- Fiji 64-bit
Once the CUDA toolkit is installed a computer restart might be required before stitching can take advantage of your GPU.
- Download CUDA toolkit v6.5 for the appropriate operating system version CUDA Toolkit 6.5
- Install the CUDA Tookit
- For Windows: Launch the executable and complete the installation
- For MacOSX: Toolkit MacOSX installation guide
- For Linux: Toolkit Linux installation guide
- Restart your computer
- Open Fiji and run MIST plugin
- CUDA option should be available in the MIST plugin.
- My image filename pattern does not work
- MIST encountered an Out of Memory problem
- What image types can MIST handle
- My stitching results look incorrect
- What if the output stitched image is too big to be saved
- I installed the CUDA toolkit, but I am getting a CUDA Exception
- The FFTW version does not work
- My stitching performance is not good
- I have a GPU in my machine but it is not showing up
- How to stitch from metadata
- Stitching Time-Slices
- Stitching Multiple Channels
- Stitching using a Macro Script
- Submitting issue or bug report
The Filename Pattern is used to match specific image files within the Image Directory.
There are two types of Filename Pattern, Sequential and Row-Column, both of which can handle Timeslices.
MIST adopts the ImageJ/Fiji notation of using curly brackets "{" and "}" to denote a block of characters that are replaced with numbers. The block "{ppp}", where the character "p" is a placeholder for a number digit, represents numbers from 000 to 999. Each letter between two curly brackets is replaced with a digit between 0 and 9.
Sequential Filename Pattern Types have only one set of curly brackets "{}" that denotes the image's position in the image grid. The special character "p" must be used between the curly brackets. Therefore, a valid sequential filename pattern must have one set of curly brackets "{p}" with at least one "p" character between the curly brackets.
Examples:
- Img_pos001.tif = Img_pos{ppp}.tif
- Img0001.png = Img{pppp}.png
- ImageName01.tif = ImageName{pp}.tif
- If a dataset has a sequence of 789 files
- common name "Dataset01_"
- with a 6 digit numbering (ex. Dataset01_000001.tif)
- Then a valid filename pattern is: "Dataset01_000{ppp}.tif" or "Dataset01_{pppppp}.tif"
Row-column Filename Pattern Types have two sets of curly brackets "{r}" "{c}". One block denotes the image's row index within the image grid and uses the special character "r". The other block denotes the image's column index within the image grid and uses the special character "c". For a valid row-column Filename Pattern there must be one "{r}" block with at least one "r" between the curly brackets and one "{c}" block with at least one "c" between the curly brackets.
Examples:
- Img_row01_col01.tif = Img_row{rr}_col{cc}.tif
- Img_r0001_c0001.png = Img_r{rrrr}_c{cccc}.png
- ImageName_001_001.tif = ImageName_{rrr}_{ccc}.tif
The usage of "{p}" or "{r}" and "{c}" must match the Filename Pattern Type selected. If the Filename Pattern contains "{p}" the sequential Filename Pattern Type must be selected. If the Filename Pattern contains "{r}" and "{c}" then row-column Filename Pattern Type must be selected.
MIST can also handle stitching a series of independent 2D image grids. For example a time-lapse series of image grids. The time-slice stitching is controlled by an additional set of curly brackets in the Filename Pattern with the "{ttt}" special text. Setting the Timeslices field in the input tab of the MIST GUI controls which of the available time-slices will be stitched. The special text "{ttt}" must be used regardless of whether the independent 2D image grid are time slices of z stack slices.
Examples:
- Img_pos001_time01.tif = Img_pos{ppp}_time{tt}.tif
- Img_r0001_c0001_t001.png = Img_r{rrrr}_c{cccc}_t{ttt}.png
- ImageName_001_001_01.tif = ImageName_{rrr}_{ccc}_{tt}.tif
- Dataset01_000001.tif = Dataset{tt}_000{ppp}.tif
There are four different execution modes MIST can run:
The mode selection exists in the Advance options tab in the GUI. Auto mode uses the FFTW mode if it can find the required native libraries and the Java mode if not.
The FFTW and CUDA modes are more memory intensive than the Java mode.
If you are having memory problems first check to see whether you are using 32bit or 64bit computation by looking under the advanced tab and de-selecting the "Use Double Precision?" checkbox.
MIST will attempt to avoid out of memory problems by detecting the available memory and adjusting the execution accordingly. MIST will first attempt to store all the image tiles in the image grid in memory to prevent spending time re-reading. If there is not enough memory to do this, it will enable the releasing of image tiles which will result in holding just the memory pool size images in memory and re-reading image tiles as required. If there is still not enough memory, MIST will lower the number of compute threads being used to reduce the size of the required memory pool. If all these strategies fail, MIST will run the stitching using a sequential Java version which requires enough memory to hold two images and two FFT data arrays.
The memory required by Java mode can be computed using the following formula, given the following four inputs.
- w = Image Grid Width
- h = Image Grid Height
- n = Number Threads Used
- s = Image Tile Size in MB
Memory Required (MB) = 6s(min(w,h) + n + 2) + 4s
If the Java mode does not have enough memory, try increasing the amount of memory Java is allowed to use.
If MIST is being run through ImageJ/Fiji there is a menu option controlling the amount memory it is allowed to use. Select the menu option "Edit" >> "Options" >> "Memory & Threads ..." which opens a dialog box containing an option "Maximum memory:" in MB. Increase this value and select "Ok". A reasonable value is about 75% of the available system memory. Restart Fiji to apply the new memory limit and re-launch MIST to see if the memory problem persists.
The Fiji FAQ has a question related to managing memory in Fiji.
For more advanced Fiji memory management information see the Java Options page maintained by the Fiji developers.
The memory required by FFTW mode can be computed using the following formula, given the following four inputs.
- w = Image Grid Width
- h = Image Grid Height
- n = Number Threads Used
- s = Image Tile Size in MB
Memory Required (MB) = 10s(min(w,h) + n + 2) + 32sn
The FFTW mode has a memory pool that holds two types of image data, the 16bit image pixel data, and the double complex FFT data (or single complex if not using 64bit computation). The size of the memory pool is determined by the size of the image grid being stitched. Memory pool size = min(gridHeight, gridWidth) + 2 + numWorkThreads. MIST iterates over the image grid in a pattern that allows the reuse of the FFT data to prevent requiring the re-computation of any FFT. There is a potential additional numWorkThreads copies of the image pixel data if the input images are not 16bit and a conversion is required from the original format to 16bit.
Unlike the Java mode, FFTW allocates native system memory in addition to memory within the JVM. When determining the available memory, MIST relies on the available JVM memory as an estimation of the available system memory.
The memory required by CUDA mode can be computed using the following formula, given the following five inputs.
- w = Image Grid Width
- h = Image Grid Height
- n = Number Threads Used
- s = Image Tile Size in MB
- g = Number of GPUs Used
CPU Memory Required (MB) = 2s(min(w,h) + n + 2) + 24sg
GPU Memory Required (MB) = 8s(min(w,h) + n + 2) + 24s + 48s
The CUDA mode has two types of memory, pinned memory that exists on both the GPU and the CPU and GPU resident memory. In addition to the pinned memory, the CUDA version needs enough CPU side memory to hold the pixel data for each image tile in the memory pool. For pinned memory, there are two double (or singel) precision arrays the size of the images, and two integer arrays the size of the images. This memory is per GPU, not per memory pool element. The GPU resident memory required consists of four double (or single) precision arrays the same size of the images and one double (or single) complex array the size of the image per memory pool element. The size of the memory pool is defined by the size of the image grid, min(gridHeight, gridWidth) + 2. On the CPU side, one image tile (at 16bit precision) needs to be stored per memory pool element.
Unlike the Java mode, CUDA allocates native memory on the GPU in addition to memory within the JVM.
Any image type that Fiji/ImageJ can read. MIST uses the ImageJ image reader contained within Fiji/ImageJ. The relevant ImageJ image types and formats documentation can be found online in the ImageJ documentation. ImageJ Documentation on Image Types.
The following are possible reasons for incorrect looking stitching:
- Incorrect Input Parameters
- Unexpected Image Overlap Variability
- Bad Stage Repeatability Estimate
- Noisy Input Images
The best way to verify the input parameters is by previewing the image grid using zero percent overlap between images. This allows you to visually check that neighboring regions look like they should overlap, and that the arrangement of images is reasonable.
To preview a zero percent overlap mosaic go to the Output tab in the GUI and select Preview Mosaic With No Overlap. Visually ensure that the image grid looks properly organized. Check for corresponding image features between regions that should overlap. Common errors include an incorrect Starting Point, Direction, Grid Width, or Grid Height.
If the zero percent overlap is taking too long to generate you can define a sub-grid using the Subgrid tab and the preview will only be generated for the subgrid.
The image tiles must be arranged in a rectangular grid and overlap with each other. The advanced parameter percent Overlap Uncertainty (default: 3%) reflects the uncertainty in the overlap between image tiles.
Set the Overlap Uncertainty to reflect the variability in the overlaps between images in your dataset.
MIST estimates the Stage Repeatability of the mechanical XY-Stage used to acquire the image grid.
The estimated repeatability is printed in the Log windows as well as in the statistics file saved in the Metadata Directory.
If MIST estimates an unreasonably large Stage Repeatability it is advisable to limit it to a value you know is reasonable for your stage. Conversely, if MIST estimates an unreasonably small Stage Repeatability it is advisable to increase it, allowing MIST to search a larger space trying to find the optimal translations.
Stage Repeatability is measured in pixels.
If the images being stitched contain a high level of noise, MIST can have trouble stitching.
Three potential solutions:
-
Enable double precision computation within the Advanced tab. This is unlikely to solve the problem, but it will not decrease the stitching accuracy. Normally double precision computation is left off because it requires more memory without providing much (if any) accuracy increase.
-
Set the advanced parameter Number of FFT Peaks (default: 2) which determines how many potential translations are tested looking for an optimal one. In noisy datasets increasing this value can improve the stitching at the cost of compute time.
-
Pre-filter the images to remove the noise before stitching. Since there are many varieties of de-noising filters, the selection and application of a reasonable one is up to the user.
If pre-filtering is used it is possible to generate a stitched image from the original images using the global positions computed from the filtered images. This is done by first stitching the filtered images (to generate the global positions metadata file) and then assembling together the stitched image from the original images and the global positions from the filtered images. See the question on how to stitch from metadata for details on how to perform stitching from metadata.
The developers are interested in problematic data sets. Issues with stitching can be submitted to nist-mist@nist.gov or MIST Github Issues Page.
MIST is written in Java which has a maximum size for any array. The ImageJ image writer that MIST relies on uses a Java array to store the pixel data imposing a limit on the size of images that can be saved.
If the number of pixels in the output stitched image exceeds 2147483647 (231-1) then the stitched image cannot be saved.
In other words, the image height in pixels multiplied by the image width in pixels must be less than 2147483647.
If the resulting stitched image cannot be saved using MIST there are two ways to get around this. The first is to stitch a sub-gid using the sub-grid options tab. This will stitch a smaller region and if that region is smaller than the maximum size then that image can be saved. The other option is to stitch the full grid without saving or displaying the resulting stitched image. This will allow MIST to stitch the images and generate the global positions metadata file. This file contains the global positions of every image in the grid. It can then be parsed and assembled using another software package.
Installing the CUDA toolkit might require a restart after the installation completes. If you attempt to run the MIST CUDA stitching before performing the restart you might get a CUDA Initialization error. If this happens restart your machine to allow the CUDA device to be initialized when MIST starts.
Alternatively, ensure that you have the correct version of the CUDA toolkit installed. See the Install Guide for details.
If the FFTW Stitching Program type does not work the most likely reason is MIST cannot find the required library file.
FFTW uses compiled native libraries to perform fast Fourier Transforms. Without these native libraries the FFTW version cannot run.
Details are available by operating system:
For Windows the correct FFTW library file should have been included with the plugin installation. To confirm this navigate to the folder where Fiji is installed. Then confirm the file "libfftw3.dll" exists in the path "lib" >> "fftw" in the directory Fiji is installed. The file might not be named exactly "libfftw3.dll", however it will always end in ".dll". If that file exists then confirm that MIST is pointing at that location in the advanced parameters tab. Open MIST, select the Advanced tab, select the FFTW Stitching Program type. Verify that FFTW Library File contains the path to the "libfftw3.dll" file you confirmed exists. Ensure the path does not just point to the correct folder, make sure that it points right at the library file. For example, "C:\Fiji.app\lib\fftw\libfftw3.dll".
For OSX (Mac) you must install FFTW before you are able to use the FFTW Stitching Program type. See the FFTW installation guide for instructions on how to install FFTW.
Once FFTW in installed ensure that the FFTW Library File parameter under the MIST Advanced parameters tab points to the location where FFTW was installed. The library file should end with ".dylib". The default location for the FFTW library is in "/usr/local/lib".
For Linux you must install FFTW before you are able to use the FFTW Stitching Program type. See the FFTW installation guide for instructions on how to install FFTW.
Once FFTW in installed ensure that the "FFTW Library File" parameter under the MIST advanced parameters tab points to the location where FFTW was installed. The library file should end with ".so". The default location for the FFTW library is in "/usr/local/lib".
There are many factors that can impact the performance of the MIST stitching:
The largest factor in the stitching performance is where the images are located.
MIST is designed with the expectation that the image to be stitched are stored on a local internal hard drive on the computer performing the stitching.
When possible copy the images to be stitched to a local hard drive before performing stitching.
Other image location options are included here ranked from slowest to fastest:
- Network hard drive connected using LAN (Ethernet)
- Any external drive connected using USB
- Local hard drive
- Local SSD
If the images are not local, read time will dominate the stitching time.
If the images are stored on a local disk, the next performance bottleneck is a lack of system memory.
If the computer performing the stitching does not have enough memory to hold all image tiles in memory, tiles will be release and re-read as required slowing down the computation.
When possible MIST will trade memory usage for compute time, increasing the required memory to lower the compute time. MIST is able to stitch your data even with low memory, but it will just be slow. The ideal case is when the computer has enough memory to store all image tiles in memory to allow MIST to read each image tile once and only once.
If memory is not an issue then the next bottleneck is the CPU.
Modern CPUs often have many cores and it might take several threads of compute work to saturate the CPU. MIST has an advanced parameter that controls how many CPU Worker Threads are used. By default this value is set to the number of cores available on the computer in an attempt to saturate the CPU by providing it enough work. Increasing the number of CPU Worker Threads beyond the default is strongly discouraged. The number of CPU Worker Threads should only be used to artificially throttle the CPU.
If the fully saturated CPU is not providing the performance you require a GPU can be added to accelerate the MIST stitching computation.
One of the MIST Stitching Program types is CUDA, which utilizes an NVidia GPU to accelerate the computation. The GPU needs to be CUDA compute capable 2.0 or greater. Also only NVidia GPUs will work as MIST relies on the CUDA toolkit which only works with NVidia GPUs. It is recommended that any GPUs have at least 1GB of graphics memory, but the actual required memory depends on the size of the image grid. For larger grids, 2GB or 4GB of graphics memory might be required. See the question MIST encountered an Out of Memory problem, specifically the CUDA Mode subsection for details on the GPU memory required.
Under the MIST Advanced parameters tab select the CUDA Stitching Program type. Press the button labeled Execute Device Query to call the CUDA device query and find which GPUs the CUDA system can find connected. If you expect to see a GPU which does not show up, check that you have the proper version of the CUDA toolkit is installed. See CUDA Install Guide for directions on how to install the CUDA toolkit version that is required. If the proper toolkit is installed and the GPU does not show up, the GPU might not have the required compute capability (compute capability of 3.0, Kepler architecture or newer), or it might not operate with CUDA in the manner MIST is expecting. Debugging GPU compatibility is beyond the scope of this FAQ.
To stitch from metadata there must be a previous execution of MIST that generated the metadata the current stitching will use. In the previous run, MIST output several metadata files into the directory indicated by the Metadata Directory specified in the Output tab. To setup the new stitching from metadata, enter the new image grid information into the Input tab. Turn the Assemble From Metadata checkbox on. This will enable the Global Positions File box. Enter into Global Positions File the filepath to the global positions metadata file from which you want to stitch. The global positions file file contains the final positions of each individual image tile in the output stitched image and those positions are used to assemble the new images into a mosaic.
MIST is designed to stitch 2D image grid datasets. It does not address volumetric or 3D stitching which requires a system to identify and correlate features across a third dimension.
However, MIST has the facility for handling a sequence of independent image grids. This ability to iterate over a series of independent image grids enables both time-lapse stitching, as well as z-stack stitching. Since each stitched image grid is independent, MIST does not care whether it is iterating over time-slices, or z-stacks.
See My image filename pattern does not work for details on how to setup the Filename Pattern.
To control which time-slices are stitched there is a field in the Input tab called Timeslices which takes a comma separated list and/or number range to specify the time-slice numbers you want MIST to stitch. For example, the Timeslices input could be "1-25,35,45" which would stitch timeslices 1 through 25, 35, and 45.
Any series of independent image grids can be input into MIST as time-slices in order to stitch them.
MIST does not currently support the direct stitching of multiple image channels.
However, by taking advantage of the assemble from metadata functionality one can perform the same task. If you have a dataset with two channels that you want to stitch, first select which channel to use for stitching. Setup and run MIST for this channel. To assemble the second channel using the same translations the first channel generated, re-launch MIST. In the input tab update the Filename Pattern to reflect the second channel. Select Assemble From Metadata. This will enable the Global Positions File text box. Enter into the Global Positions File box the filepath to the global positions metadata file to use to assemble the new channel.
In the output tab keep the same Output Directory and Metadata Directory. Change the output filename prefix to something different than what was used for the first channel. Select Begin Stitching and MIST will use the global positions file selected by the Global Positions File text box to assemble the new channel into a stitched image.
ImageJ/Fiji has the ability to record and run Macro script to automate tasks. See an Introduction to Macro Programming for a general overview of macro programming in Fiji.
To create a MIST stitching macro use the macro recorder.
Select "Plugins" >> "Macros" >> "Record"
This will open a window showing the macro script version of any commands you run in Fiji.
Open the MIST plugin and setup the parameters to run your experiment and select "Begin".
The macro recorder should capture the information required to run MIST from a macro script. The captured command can be copied to a macro script and modified to run MIST from a macro.
A motorized mechanical XY-Stage is used to move a biological sample with respect to the microscope’s optical imaging system. This movement is done by two independent stepper motor linear actuators, one for each direction.
It is important to understand the mechanical limitations of a stepper motor linear actuator mainly with regards to its accuracy and repeatability.
Accuracy: Actuator motion can only be as true as the mechanical components permit. The accuracy is the actuator’s ability to provide precise increments of motion along its axis. In a mechanical system, this primarily concerns the lead screw as well as the feedback device (encoder, linear scale etc.). Lead accuracy of the screw, resolution of encoders, and ability of the controller must combine to produce the desired accuracy. In most microscopes the accuracy is very high and can be calibrated.
Repeatability: Repeatability is the ability of a device to reach a specific location multiple times. It does not take into account the trueness of the position but rather the ability to go back to that position. Many times the actuator will follow a slightly bowed or twisted path due to imperfect construction. The repeatability is the measure of uncertainty relative to a given actuator movement. The repeatability cannot be calibrated, but it can be measured for any microscope stage. Each time the stage controller requests a position value, the actual resulting position is within repeatability of the requested value.
In a grid tiling, the positions (x, y), that the stage will go to, have an uncertainty equal to the stage repeatability (x ± r, y ± r). However, translations (dx, dy) computed in the vertical or horizontal directions between consecutive tiles are differences between respective positions. Thus, the uncertainty on the computed translation values is 2*repeatability(dx ± 2r, dy ± 2r).
When computing vertical translations between two consecutive tiles, the dx and dy components will correspond to the projection of the stage displacement on the camera coordinate systems (which might have a fixed rotational angle) with an uncertainty equal to 2*repeatability. We use this information to estimate the stage repeatability from the computed translations.
The Stage Repeatability value can be entered into MIST under the advanced parameters tab of the GUI. MIST expects a Stage Repeatability values in pixels.
The code for MIST is being stored and managed on Github which has the functionality to report and track issues. To report a bug go to the MIST Github page and select "Issues" on the right hand side of the page. Or go directly to the Issues page. To create a new issue select the green "New Issue" button on the top right side of the page.
When posting in issue please be as specific as you can and if possible include the following data generated by MIST from the stitching experiment that encountered the error.
- The contents of the Log window in Fiji.
- done by clicking the menu option "File" >> "Save As" on the Log window
- The relevant metadata files
- statistics file (important)
- global positions file
- relative positions file
- relative positions no optimization file
If you have questions or issues regarding MIST feel free to email the developers at: nist-mist@nist.gov