WO2017004555A1 - System, method and computer-accessbile medium for multi-plane imaging of neural circuits - Google Patents

System, method and computer-accessbile medium for multi-plane imaging of neural circuits Download PDF

Info

Publication number
WO2017004555A1
WO2017004555A1 PCT/US2016/040753 US2016040753W WO2017004555A1 WO 2017004555 A1 WO2017004555 A1 WO 2017004555A1 US 2016040753 W US2016040753 W US 2016040753W WO 2017004555 A1 WO2017004555 A1 WO 2017004555A1
Authority
WO
WIPO (PCT)
Prior art keywords
radiation
slm
sample
exemplary
arrangement
Prior art date
Application number
PCT/US2016/040753
Other languages
French (fr)
Inventor
Rafael Yuste
Darcy S. PETERKA
Weijian Yang
Original Assignee
The Trustees Of Columbia University In The City Of New York
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by The Trustees Of Columbia University In The City Of New York filed Critical The Trustees Of Columbia University In The City Of New York
Priority to US15/741,435 priority Critical patent/US10520712B2/en
Publication of WO2017004555A1 publication Critical patent/WO2017004555A1/en
Priority to US16/730,344 priority patent/US20210033836A1/en

Links

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/06Means for illuminating specimens
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/06Means for illuminating specimens
    • G02B21/08Condensers
    • G02B21/082Condensers for incident illumination only
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/16Microscopes adapted for ultraviolet illumination ; Fluorescence microscopes
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/36Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365Control or image processing arrangements for digital or video microscopes
    • G02B21/367Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B26/00Optical devices or arrangements for the control of light using movable or deformable optical elements
    • G02B26/08Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
    • G02B26/10Scanning systems
    • G02B26/101Scanning systems with both horizontal and vertical deflecting means, e.g. raster or XY scanners
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B2207/00Coding scheme for general features or characteristics of optical elements and systems of subclass G02B, but not including elements and systems which would be classified in G02B6/00 and subgroups
    • G02B2207/114Two photon or multiphoton effect
    • GPHYSICS
    • G02OPTICS
    • G02FOPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS
    • G02F1/00Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics
    • G02F1/01Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the intensity, phase, polarisation or colour 
    • G02F1/03Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the intensity, phase, polarisation or colour  based on ceramics or electro-optical crystals, e.g. exhibiting Pockels effect or Kerr effect
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10056Microscopic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10064Fluorescence image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30016Brain
    • G06T5/70

Definitions

  • the present disclosure relates generally to multi-plane imaging, and more specifically, to exemplary embodiments of an exemplary system, method and computer-accessible medium for, e.g., simultaneous multi-plane imaging of neural circuits.
  • Optical imaging methods aim to capture this activity, with recent progress now facilitating the functional imaging of nearly the entire brain of an intact transparent organism, the zebra fish, with cellular resolution. ⁇ See, e.g., Reference 1). In scattering tissue, where nonlinear microscopy can be beneficial ⁇ see, e.g., References 23, 63, and 69), progress toward imaging large pools of neurons has been slower.
  • a single beam can be serially scanned in a continuous trajectory across the sample with galvanometric mirrors, in a raster patterns or with a specified trajectory that intersects targets of interest along the path. This means that the imaging can be serial and thus slow.
  • a difficulty in imaging can be linked to expanding the volumetric extent of imaging, while maintaining high temporal resolution and high sensitivity. ⁇ See, e.g., References 2 and 3). This can generally be linked to the inverse relationship between volume scanned, and the signal collected per voxel, at a fixed resolution.
  • an exemplary device can include, for example, a radiation source(s) configured to generate a first radiation(s), a spatial light modulator (SLM) arrangement(s) configured to receive the first radiation(s) and generate a second radiation(s) based on the first radiation(s), and a galvanometer(s) configured to receive the second radiation(s), generate a third radiation(s) based on the second radiation(s), and provide the third radiation(s) to a sample(s).
  • a radiation source(s) configured to generate a first radiation(s)
  • SLM spatial light modulator
  • the SLM arrangement(s) can include, for example, a SLM and a pre-SLM afocal telescope configured to resize the first radiation(s) to match an area of the SLM.
  • the SLM arrangement(s) can further include a plurality of folding mirrors configured to redirect the first radiation(s) to the pre-SLM, and a post SLM afocal telescope configured to resize the second radiation(s) to match a size of an acceptance aperture of the galvanometer(s).
  • the SLM arrangement(s) can also further include a broadband waveplate(s) located between the pre-SLM afocal telescope and the SLM.
  • the broadband waveplate(s) can be configured to rotate a polarization of the first radiation(s) to cause the radiation(s) to be parallel with an active axis of the SLM.
  • the SLM arrangement(s) can be configured to split the first radiation(s) into the radiation beamlets which are the second radiation(s).
  • the SLM arrangement(s) can be further configured to independently dynamically control each of the radiation beamlets.
  • the SLM arrangement(s) can split the first radiation(s) into the radiation beamlets by imprinting a phase profile across the first radiation(s).
  • the galvanometer(s) can be further configured to direct each of the radiation beamlets to a different area of the sample(s).
  • the galvanometer(s) can direct each of the radiation beamlets to a different plane of the sample(s).
  • the radiation source(s) can be a laser source(s).
  • a computer processing arrangement can be configured to generate an image(s of the sample(s) based on a plurality of resultant radiations received from the sample(s) that can be based on the radiation beamlets.
  • a first number of the resultant radiations can be based on a second number of the radiation beamlets.
  • the second number of the radiation beamlets can be based on a third number of the planes of the sample(s).
  • the computer processing arrangement can be further configured to generate a third number of images of the sample(s) based on the resultant radiations.
  • the image(s) can include a plurality of images.
  • the computer processing arrangement can be further configured to generate a multiplane image(s) based on the images.
  • the multiplane image(s) can be generated by interleaving the images into the multiplane image(s).
  • the computer processing arrangement can be further configured to correct brain motion artifacts in the images based on a pyramid procedure.
  • a pocket cell(s) can be located between the radiation source(s) and the SLM arrangement(s), which can be configured to modulate an intensity of the first radiation(s).
  • a computer processing arrangement can be provided, which can be configured to generate an image(s) of the sample(s) based on a fourth radiation(s) received from the sample(s) that can be based on the third radiation(s).
  • an exemplary method can include, for example, generating a radiation(s), providing the radiation(s) to a spatial light modulator (SLM) arrangement(s), splitting the radiation(s) into a plurality of radiation beamlets using the SLM arrangement(s), and directing the radiation beamlets to a sample(s) using a galvanometer(s).
  • a computer hardware arrangement can be used to generate an image(s) of the sample(s) based on a resultant radiation received from the sample(s) that can be based on the radi ati on b eaml ets .
  • the radiation(s) can be generated using a laser(s).
  • Each of the radiation beamlets can be independently dynamically controlled using the SLM arrangement(s).
  • the SLM arrangement(s) can split the radiation(s) into the plurality of radiation beamlets by imprinting a phase profile across the radiation(s).
  • the SLM arrangement(s) can include a SLM(s).
  • Each of the radiation beamlets can be directed to a different area of the sample(s).
  • Each of the radiation beamlets can be swept across the respective different area of the sample(s) using the galvanometer(s).
  • each of the radiation beamlets can be directed to a different plane of the sample(s).
  • a plurality of resultant radiations can be received from the sample(s) that can be based on the radiation beamlets.
  • a first number of the resultant radiations can be based on a second number of the radiation beamlets.
  • the second number of the radiation beamlets can be based on a third number of the planes of the sample(s).
  • a third number of images of the sample(s) can be generated based on the resultant radiations.
  • a plurality of images can be generated based on the resultant radiations.
  • a multiplane image(s) can be generated based on the images.
  • the multiplane images can be generated by interleaving the images into the multiplane image(s).
  • Brain motion artifacts can be corrected in the images based on a pyramid procedure.
  • Figure 1 A is an exemplary diagram of an exemplary SLM Two-photon Microscope according to an exemplary embodiment of the present disclosure
  • Figures IB and 1C are exemplary diagrams of axial and lateral dual plane imaging according to an exemplary embodiment of the present disclosure
  • Figure ID is a set of exemplary images of two-photon structural imaging of a shrimp (e.g., artemia nauplii) at different depths of the sample according to an exemplary embodiment of the present disclosure
  • Figure IE is a set of exemplary images of software based SLM focusing of the same shrimp as in Figure ID according to an exemplary embodiment of the present disclosure
  • Figure IF is an exemplary image of the sum of all the images at the seven planes shown in Figure ID according to an exemplary embodiment of the present disclosure
  • Figure 1G is an exemplary image of the sum of all the images at the seven planes shown in Figure IE according to an exemplary embodiment of the present disclosure
  • Figure 1H is an exemplary image of seven-axial-plane imaging of the same shrimp as in Figures 1D-1G according to an exemplary embodiment of the present disclosure
  • Figure II is an exemplary image of seven-axial-plane imaging using the SLM to increase the illumination intensity only for the 50 ⁇ plane according to an exemplary embodiment of the present disclosure
  • Figure 2A is an exemplary diagram of the exemplary in-vivo experiment imaging the VI of the mouse according to an exemplary embodiment of the present disclosure
  • Figures 2B and 2C are exemplary images of the temporal standard deviation image of the sequential single plane recording of mouse VI at a depth of about 280 ⁇ from the pial surface according to an exemplary embodiment of the present disclosure
  • Figure 2D is an exemplary image of some of the illustrations from Figures 2B and 2C according to an exemplary embodiment of the present disclosure
  • Figure 2E is an exemplary image of the temporal standard deviation image of the simultaneous dual plane recording of the two fields of view according to an exemplary embodiment of the present disclosure
  • Figure 2F is an exemplary signal diagram of the extracted AF/F traces of the selected ROIs from the two field of views according to an exemplary embodiment of the present disclosure
  • Figure 2G is an exemplary signal diagram of extracted AF/F traces of the same ROIs shown in Figure 2F from the simultaneous dual plane recording according to an exemplary embodiment of the present disclosure
  • Figure 2H is an exemplary signal diagram of a zoomed in view of the normalized AF/F traces in the shaded area in Figure 2G according to an exemplary embodiment of the present disclosure
  • Figures 3 A and 3B are exemplary images of the temporal standard deviation image of the sequential single plane recording of mouse VI at a depth of about 170 ⁇ (e.g., layer 2/3) and depth of about 500 ⁇ (e.g., layer 5) from the cortical surface according to an exemplary embodiment of the present disclosure;
  • Figure 3C is an exemplary image using the exemplary system according to an exemplary embodiment of the present disclosure.
  • Figure 3D is an exemplary image of the temporal standard deviation image of the simultaneous dual plane recording of the two planes shown in Figures 3 A and 3B according to an exemplary embodiment of the present disclosure
  • Figure 3E is an exemplary signal diagram of the extracted AF/F traces of 20 ROIs out of 350 from the two planes from the sequential single plane recording according to an exemplary embodiment of the present disclosure
  • Figure 3F is an exemplary signal diagram of extracted AF/F traces of the same ROIs shown in Figure 3E from the simultaneous dual plane recording according to an exemplary embodiment of the present disclosure
  • Figures 3G-3I are zoomed views of the exemplary signal diagrams of AF/F traces in the shaded area in Figures 3E and 3F according to an exemplary embodiment of the present disclosure
  • Figures 4A-4C are exemplary signal diagrams of the source separation of the fluorescent signal from spatially overlapped ROIs in the dual plane imaging shown in Figures 3 A-3I according to an exemplary embodiment of the present disclosure
  • Figure 4D is an exemplary signal diagram of the ROI contour map showing the ROI locations in Figures 4A-4C according to an exemplary embodiment of the present disclosure
  • Figure 4E is an exemplary chart illustrating the correlation coefficient between the AF/F extracted from MF and NOL for a total of 250 ROIs according to an exemplary embodiment of the present disclosure
  • Figure 4F is an exemplary graph illustrating the signal-to-noise ratio between the AF/F extracted from NMF and the raw AF/F extracted from NOL for a total of 250 ROIs according to an exemplary embodiment of the present disclosure
  • Figure 5A is a set of exemplary signal diagrams of normalized AF/F traces of selected 150 ROIs in the dual axial plane imaging in Figures 3A-3I according to an exemplary embodiment of the present disclosure
  • Figure 5B is an exemplary chart illustrating the inter-laminar correlation map of activity between neurons in L2/3 and L5 according to an exemplary embodiment of the present disclosure
  • Figure 5D is an exemplary chart illustrating the activity correlation coefficient extracted from Figure 5B versus the lateral distance of the corresponding pairs of ROIs according to an exemplary embodiment of the present disclosure
  • Figure 6A is an exemplary signal diagram of normalized AF/F traces for selected ROIs with strong response to drifting grating visual stimulation, recorded with simultaneous dual plane imaging according to an exemplary embodiment of the present disclosure
  • Figure 6B is an exemplary diagram and an exemplary chart illustrating of the response of the ROIs to the drifting grating in visual stimulation according to an exemplary embodiment of the present disclosure
  • Figure 6C is an exemplary diagram and an exemplary chart illustrating the response of the ROIs to the drifting grating in visual stimulation for ROIs located at 450 ⁇ depth from cortical surface according to an exemplary embodiment of the present disclosure
  • Figure 6D1 is an overlaid temporal standard deviation image of the sequential single plane recording of the 200 ⁇ plane and 450 ⁇ plane according to an exemplary embodiment of the present disclosure
  • Figure 6D2 is an exemplary image of extracted ROI contours from the two planes with a Scale bar of 50 ⁇ according to an exemplary embodiment of the present disclosure
  • Figure 6E is an exemplary diagram of the evoked responses of the ROIs with spatial lateral overlaps from the two planes according to an exemplary embodiment of the present disclosure
  • Figure 7A is an exemplary graph illustrating the reflected phase of the SLM for a wavelength of 940 nm versus the applied driving pixel value to the SLM according to an exemplary embodiment of the present disclosure
  • Figure 7B is an exemplary graph illustrating SLM two-photon fluorescence efficiency with different defocusing length, measured from the fluorescence emitted from Rhodamine 6G with two-photon excitation according to an exemplary embodiment of the present disclosure
  • Figure 7C is an exemplary graph illustrating SLM lateral deflection efficiency, measured from the optical power at the back aperture of the objective according to an exemplary embodiment of the present disclosure
  • Figure 7D is an exemplary graph illustrating calculated SLM lateral deflection efficiency according to an exemplary embodiment of the present disclosure
  • Figure 7E is an exemplary image of 12 spots generated by SLM simultaneously imaged on a CCD according to an exemplary embodiment of the present disclosure
  • Figure 7F is an image of 12 spots generated by about SLM simultaneously imaged on CCD with power compensation for each spot for SLM lateral deflection efficiency according to an exemplary embodiment of the present disclosure
  • Figures 8A and 8B are exemplary ROI contour and calcium signals according to an exemplary embodiment of the present disclosure.
  • Figure 9A is an exemplary chart illustrating the correlation coefficient between the AF/F extracted from C MF and NOL for a total of 250 ROIs according to an exemplary embodiment of the present disclosure
  • Figures 9B-9G are exemplary charts illustrating ROIs with various correlation coefficient between CNMF and NOL according to an exemplary embodiment of the present disclosure
  • Figures 10A-10H are exemplary images and signal charts of three plane imaging, on mouse VI at depth of 170 ⁇ , 350 ⁇ and 500 ⁇ from pial surface according to an exemplary embodiment of the present disclosure
  • Figures 10I-10O are exemplary images and signal charts of three plane imaging, on VI at depth of 130 ⁇ , 430 ⁇ and 640 ⁇ from pial surface, extending a simultaneous imaging depth over 500 ⁇ according to an exemplary embodiment of the present disclosure
  • Figure 11 A is a set of exemplary images of SLM switching between two sets of dual plane imaging on mouse VI according to an exemplary embodiment of the present disclosure
  • Figure 1 IB is an exemplary chart illustrating switching time between State 1 and State 2, measured from fluorescent signal emitted from Rhodamine 6G according to an exemplary embodiment of the present disclosure
  • Figure 11C is an exemplary diagram of a volumetric imaging procedure using time- multiplexed three-plane imaging according to an exemplary embodiment of the present disclosure.
  • Figure 12 is an illustration of an exemplary block diagram of an exemplary system in accordance with certain exemplary embodiments of the present disclosure.
  • An exemplary embodiment of the present disclosure can include, for example an exemplary SLM microscope coupled with a two-photon microscope, galvanometers and an SLM module.
  • Figure 1 A shows a schematic diagram of the multi-plane imaging and the exemplary SLM two-photon microscope apparatus.
  • the exemplary SLM module can include diverting the input path of the microscope, prior to the galvanometer mirrors 140, using retractable kinematic mirrors, onto a compact optical breadboard with the SLM 130 and associated exemplary components.
  • Exemplary features of the SLM module can include folding mirrors for redirection, a pre-SLM afocal telescope 115 to resize the incoming beam to match the active area of the SLM, the SLM 130 and a post-SLM afocal telescope 135to resize the beam again to match the acceptance aperture of the galvanometers 140, and to fill the back focal plane of the objective 155 appropriately.
  • the exemplary SLM 130, post-SLM telescope 135 and galvanometers 140 can be spaced such that the SLM 130 can be conjugate to the galvanometers 140, and the microscope scan lens 145, and tube lens 150, can reimage this again to the back aperture of the microscope objective.
  • This module can be coupled to 2P microscopes, and to Prairie/Bruker systems, with equal success, and similar performance.
  • the SLM 130 can be used as a flexible, programmable beam splitter that can facilitate independent dynamic control of each generated beamlet, at high speed.
  • the exemplary SLM 130 can perform this flexible beamsplitting by imprinting a phase profile across the incoming wavefront, resulting in a far field diffraction pattern yielding the desired illumination pattern.
  • each individual beamlet can sweep across its targeted area on the sample 160, generating a fluorescence that can be collected by a single pixel detector (e.g., photomultiplier tube).
  • a single pixel detector e.g., photomultiplier tube.
  • the resultant "image" can be a superposition of all of the individual images that would have been produced by scanning each separate beamlet individually ⁇ See, e.g., Figures IB and 1C).
  • the exemplary SLM can be used to generate all seven axially displaced beamlets simultaneously, and can scan them across the sample. ⁇ See, e.g., Figure 1H).
  • a Bessel or an Airy beam can be used to directly provide an extended depth of field. ⁇ See, e.g., References 19 and 59). While these beams can give a similar result about for the shrimp, or similarly transparent samples, they can provide near uniform intensity across the entire depth of field, while the exemplary approach can facilitate independent power adjustment of each depth. This is shown in Figure II where the power directed to a single plane (e.g., 50 ⁇ ) is selectively increased, and the signal, only for features at that depth, is increased.
  • This flexibility can be beneficial for inhomogeneously stained samples, and for multi- depth in-vivo imaging in scattering tissue, as is illustrated below, and may only be possible with independently configurable beamlets. Additionally, by illuminating only a select number of planes, the "density" of the resulting images can be controlled; for example, how many sections used to contribute to the final image can be controlled.
  • FIGS 2A-2H the functional imaging application of the exemplary system 205 is shown.
  • the exemplary SLM can be configured similar to that illustrated in Figure 1C, to expand the effective sampled area.
  • the beam can be laterally split, creating two beams of equal power with an on-sample separation of -300 ⁇ centered on the original field of view ("FOV").
  • FOV field of view
  • Figures 2B-2D show images of the standard deviation ("std. dev.") of intensity across the acquired time-series image sequences that result from scanning each of these displaced beams individually, and their arithmetic sum, respectively, while Figure 2E shows the std. dev. image acquired when both beams can be simultaneously scanned across the sample.
  • the lower subpanels show detected source ROIs from the images, with the element 210 and 215 reflecting the originating source FOV.
  • Figures 2F and 2G show representative extracted fluorescence time series data from the detected ROIs (e.g., 40 out of 235 shown). The same ROIs are displayed in both Figures 2F and 2G, with the same ordering, to facilitate direct comparison of the single and dual plane traces.
  • the FOVs were collected sequentially - that can be one at a time, and thus the traces 230 were collected at a different absolute time than the traces 235. Traces 240 and 245 traces shown in Figure 2F, however, were collected simultaneously.
  • the overall effective SNR can be high, in both the sequentially acquired data and in the simultaneously collected dual region data, which can facilitate events to be easily detected by exemplary automated procedures.
  • the collected multi-region image can be the arithmetic sum of the two single region images
  • the detected ROIs from the single region image can be used as strong prior knowledge for source localization. This can be leveraged to produce very good initial estimates on the likely number of independent sources, and their spatial location. This prior knowledge can be extremely useful for unmixing complex overlapping signals, and can increase the overall performance of the source extraction in the mixed images.
  • the "uniqueness" of signal recovery can be examined by looking at the ROIs that appear twice in the dual plane image, and can display identical dynamics.
  • traces 255 and 260 show the source copy generated from each beamlet, and the extremely high correlation between the extracted traces (e.g., R>0.985).
  • These duplicative sources can be easily removed from the total independent ROI count; first, because the positioning of the FOVs can be deterministically controlled, and a priori known, which regions can be the shared, and where the components will appear in the image can be determined. It can be noted that even without that such knowledge, such source ROIs can be identified by their extremely high cross-correlation.
  • the dual region image can include signals from a significantly larger total area.
  • the single region FOV was approximate exemplary 380 ⁇ x 380 ⁇ , and captured 1.45xl0 5 ⁇ 2 , while the dual region image captured signals from
  • the exemplary maximal useful lateral displacement of each beamlet from the center of the FOV approximately 150 um. ⁇ See, e.g., Figure 7C and 7D).
  • the effective frame rate of a system can be doubled by creating two laterally displaced beams, with an angular spread one half of the FOV, in the direction orthogonal to the fast axis of the galvos.
  • the displaced beams can still illuminate the entire FOV, though the total number of lines scanned can be halved, which can double the overall frame rate. Alternately, two or more small subregions enclosed within a single larger FOV can be scanned simultaneously. Under all of these paradigms, it can be noted that the splitting can facilitate increases in effective frame rate while keeping the dwell time per pixel of each region higher than what would be possible if the regions were sequentially scanned, which can increase the overall signal collected from each region. [0075] While the lateral imaging procedures can increase imaging performance, the full power of multiplexed SLM imaging lies in its ability to flexibly address axially displaced planes, with independent control of beamlet power and position.
  • An exemplary defocus aberration can be introduced to the wavefront, which can shift the beam focus away from the nominal focal plane.
  • Higher order axially dependent phase terms can be included to offset the effects of higher-order aberrations, and facilitate "prism" shifts as well, which can add flexibly by facilitating for lateral displacements.
  • the exemplary system can provide high performance, and gives approximately 500 ⁇ of axial displacement while maintaining total collected two-photon fluorescence at greater than 50% of that generated at the objective's natural focal plane. ⁇ See, e.g., Figure 7B). This can provide a considerable range for scanning, and it does so with high speed inertia-free focusing (e.g., less than 3 ms, as shown in Figure 11).
  • FIGS. 3 A and 3B show two examples of conventional "single- plane" two-photon images in mouse VI, the first approximately 170 ⁇ below the pial surface, in L2/3, and the second 500 ⁇ below the surface, in L5.
  • Figure 3E shows some representative extracted fluorescence traces from L2/3 ⁇ e.g., element 305), and L5 ⁇ e.g., element 310) (e.g., 154 and 196 total ROIs detected across the upper and lower planes, respectively).
  • the exemplary SLM was then used to simultaneously split the incoming beam into two axially displaced beams, directed to cortical depths of 170 ⁇ and 500 ⁇ , and scanned over the sample at 10 Hz. ⁇ See, e.g., Figure 3D). Because the mouse cortex can be highly scattering ⁇ see, e.g., References 17 and 33), significantly more power can be needed to image L5 than L2/3, and the power directed to each plane can be adjusted such that the collected fluorescence from each plane was approximately equal. The effective collected two photon signal can also depend on the overall efficiency of the SLM in redirecting the light to axial positions other than the designed focal plane of the microscope.
  • optical parameters of the two-photon microscope such as the bandwidth of the laser source and the effective numerical aperture ("NA") of excitation
  • NA effective numerical aperture
  • SLM specific device parameters such as the number distinct phase levels, the pixels density, and the fill factor of the device.
  • it can be beneficial to holographically deflect light over a span of approsimately500 ⁇ axially, while also maintaining clear subcellular resolution and high two- photon efficiency.
  • the exemplary measured efficiency curve can be slightly asymmetric (see, e.g., Figure 7B), and shows that the beam can be projected from 200 ⁇ beyond (e.g., deeper) the focal plane of the objective, to 300 ⁇ above (e.g., shallower) than the objective's focal plane while maintaining strong two-photon excitation.
  • This curve can be relied upon as the objective's focal plane depth to a specific position, and can be set to optimally deliver power to the chosen targeted planes for simultaneous imaging, or to compute the power used for a good signal at various multiplane combinations.
  • the L5 plane can utilize approximately six times the input power than that of the L2/3 image to match the signals.
  • the SLM efficiency for redirection can be the same for both planes, and the dual beam hologram can be computed with an approximately six-fold increase in intensity for the lower layer.
  • the dual plane image can be collected, which is shown, along with the ROIs, in Figure 3D, which can correspond very well to the arithmetic sum of the individual plane images.
  • Figure 3F thirty representative traces (e.g., out of 350 source ROIs) are shown, which illustrate spontaneous activity across L2/3 (e.g., element 315) and L5 (e.g., element 320), with a high SNR.
  • the ordering of traces is identical with that of Figure 3E, which facilitates comparison of the signals detected with the conventionally acquired single plane images.
  • the lightly shaded regions 325 in Figures 3E and 3F are enlarged and shown in Figures 3G and 3H.
  • the traces 330 and 335 can be the signals extracted from the ROI using the CNMF method, which both optimally weights individual pixels and denoises the signal, while the underlying trace 340 shows the raw signal from the ROI.
  • the raw signals show very clear events, with high SNR, and the CNMF traces can be even cleaner.
  • Further exemplary zooms of small events are shown in Figure 31, which shows expanded views of the small peaks labeled i-iv on Figures 3G and 3H.
  • a generalized biophysical model can be used to relate the detected fluorescence from a source (e.g., neuron) to the underlying activity (e.g., spiking) (See, e.g., References 61 and 62).
  • a source e.g., neuron
  • the underlying activity e.g., spiking
  • the exemplary goal then can be, given a set of pixels of time varying intensity, infer the low-rank matrix of underlying independent signal sources that generated the measured signals.
  • the non- negativity of fluorescence and of the underlying neuronal activity can be taken advantage of, and the computationally efficient constrained non-negative matrix factorization methods can be used to perform the source separation; thus the label of C MF.
  • the exemplary procedure can be initialized with the expected number of sources (e.g., the rank), along with the nominal expected spatial location of the sources as prior knowledge, as identified by running the procedure on the previously acquired single plane image sequences.
  • the expected number of sources e.g., the rank
  • the nominal expected spatial location of the sources as prior knowledge, as identified by running the procedure on the previously acquired single plane image sequences.
  • the complexity, and number of overlapping sources can be significantly less than the multiplane images, and the procedure works very well for identifying sources without additional guidance.
  • Figures 4A-4F effectiveness of this exemplary procedure, as applied to multiplane imaging, is shown in Figures 4A-4F.
  • This can also be compared against the exemplary "best" human effort at selecting only the non, or minimally overlapping, pixels from each source, and against independent component analysis ("ICA"), which has previously proved very successful in extracting individual sources from mixed signals in calcium imaging movies(See, e.g., Reference 39).
  • Figures 4A-4C show progressively more complex spatial patches of the dual plane image series with Figure 4D showing the location of each patch in the overall image.
  • the overall structure shown in Figures 4A- 4C is identical. In each figure therein, the uppermost row of images show the maximum intensity projection of the time series, with the subsequent images showing, representative time points where the component sources can be independently active.
  • the leftmost column of images shows the weighted mask, labeled with the ROI selection scheme (e.g., binary mask from maximum intensity projection, human selected non-overlapped, ICA, CNMF) that can produce the activity traces presented immediately to the right of these boxes. Therefore, the pixel weighting is shown adjacent to Figure 4D.
  • the extracted CNMF traces can include both the full CNMF extracted trace 405, and for comparison, trace 410, the signal extracted considering only the CNMF produced spatially weighted ROI, without taking into account the temporal mixing of other sources into those pixels.
  • the human selected non-overlapped regions appear to select only a subset of the events seen in the combined binary mask.
  • the non-overlapped portion may only contain a few pixels, yielding a poor SNR, or those pixels may not be truly free from
  • the ICA extracted sources can then be examined. ICA can identify the sources automatically, without human intervention, and does so quickly. For cases where the number of sources in space can be low, and there can be "clean" non-overlapping pixels with high SNR (see, e.g., Figures 4A and 4B), the extracted components can be spatially consistent with the known source location (e.g., top row of images). In many cases, they can include a region of low magnitude negative weights, which can appear to spatially overlap adjacent detected sources, presumably because this can decrease the apparent mixing of signals between the components. This can lead to unphysical minor negative transients, but these can be easily ignored with simple thresholding.
  • ICA can routinely fail to identify human and CNMF identified source components, and appears to have less clean separation of mixed signals (e.g., activity traces shown in Figure 4A, and the complete failure of ICA to identify ROI2 shown in Figure 4C). Additionally, in order to maximize the performance of ICA, the acquired image can be tiled into smaller sub-images, with fewer component sources, for ICA to give reasonable performance.
  • the graph shown in Figure 9A illustrates the exemplary correlation coefficient between the AF/F extracted from CNMF and NOL for a total of 250 ROIs, for the data shown in Figures 3A-3I and 4A-4F.
  • the dashed 905 line indicates the median of the correlation coefficients.
  • the ROI contour map is also plotted.
  • Figures 9B-9G show exemplary ROIs with various correlation coefficient between CNMF and NOL.
  • the ROI is labelled as 1, and its adjacent ROIs (e.g., potential contamination sources) are shown in Figure 9 A.
  • both signals are plotted.
  • the extracted AF/F trace 910 is plotted, superimposed onto the traces extracted from CNMF.
  • the signals extracted from their adjacent ROIs are plotted in green.
  • the exemplary model can facilitate this in a
  • the identified sources can be well separated. Comparing the CNMF trace to the non-overlapped trace, a very high correspondence can be seen, especially when the respective non-overlapped source SNR can be high. As shown in Figure 4A, it is possible to see that the CNMF can be better than ICA at eliminating cross-talk between the two overlapping ROIs. This trend can be seen across all of the examples shown, and can generally be conserved over all ROIs.
  • the CNMF traces can be compared against the human selected non- overlapping traces 415 shown in Figure 4E. This exemplary graph shows the cross correlation between 250 CNMF sources, and the non-overlapped portion of that source (e.g., NOL) related to it.
  • the correlation can be high, as would be expected if the CNMF traces accurately detect the underlying source.
  • the majority of the signals show a correlation coefficient of less than 0.95, and the mean correlation coefficient can be 0.91, including all outliers. It can be noted that the distribution of coefficients can be strongly asymmetric, and it can contain some notable outliers.
  • the source of the poor correlation between the NOL trace and the CNMF trace can be identified. ⁇ See, e.g., Figures 10B-10G).
  • the NOL ROI can consist only of a few pixels, and can be extremely noisy.
  • the related CNMF ROI can be significantly larger and has much less noise.
  • sources can be seen with very different activity than that extracted in the CNMF trace, but clearly are present in the NOL signal. This phenomenon can be present in all of the lower correlation components. Because of this, the graph can be interpreted in another way.
  • the fraction of cells with low correlation between the NOL and CNMF components can be exactly the sources that simple signal extraction procedures cannot cleanly extract the true underlying activity, they can be the cells that benefit from CNMF. Without CNMF, extracted signals can falsely show high correlation between sources that can be imaged into the same region in the dual plane image. In the dual plane image shown in Figure 3, it can be found that approximately 30% of the cells have correlations below 0.9.
  • the exact fraction of cells with significant contamination can depend on the sources sparsity and targeted areas, but there may always be many cells that can show some overlap, and would benefit from a high performance unmixing strategy, like CNMF.
  • FIGs 5A-6E The power of the exemplary simultaneous multiplane imaging and source separation approach can be seen in Figures 5A-6E, where interlaminar correlations can be recorded, which can be used for understanding microcircuit information flow in the brain, as well as can evoke functional responses across cortical layers.
  • Figure 5 A 150 fluorescence activity traces 505 (e.g., out of approximately 350) showing spontaneous activity in L2/3 and L5 neurons in VI, collected at 10 Hz (e.g., FOV and cells as in Figures 3A-3H) are shown.
  • 10 Hz e.g., FOV and cells as in Figures 3A-3H
  • 13 minutes of imaging over 20,000 events were detected across the population of active cells (e.g., 250 ROIs).
  • the significant correlations between cells in L2/3 and cells in L5 can be computed, and the interlaminar correlation map in Figure 5B can be displayed.
  • One of the high correlation pairs e.g., projected lateral displacement of approximately 67 ⁇
  • element 510 e.g., dark outlines
  • Figure 5D A scatterplot showing the correlation coefficients between pairs with respect to projected lateral displacement is shown in Figure 5D. It can be noted that for the spontaneous activity in this recording, little dependence in the average correlation between cells in L2/3 and L5 on lateral distance between cells (e.g., inset, Figure 5D) can be seen.
  • FIG. 10A-10H simultaneous three-plane imaging, again in mouse VI, is shown, and some representative activity traces from each of the planes is highlighted, as well as clean source separation using CNMF for a region of interest with partial spatial overlap from each of the planes can be shown. Deeper, extended, axial planes can be further
  • the two small boxes 625 on the contour image indicate two pairs of cells, with significant spatial overlap in the dual-plane image. This is illustrated in more detail in Figure 6E, where the DS of these cells is shown to ensure that the source separation procedure cleanly extracted the functional activity. It can be clear that the functional activity of even strongly overlapping cells can be fully separated, without cross contamination.
  • the exemplary procedure can be easily implemented on any microscope, with the addition of a SLM module to the excitation path, and without any additional hardware modifications in the detection path.
  • the regional targeting can be performed remotely, through holography, without any motion of the objective, which can make the exemplary procedure a strong complement to 3D two-photon activation ⁇ See, e.g., References 44, 45, and 51).
  • a critical component for determining the imaging rate can be the speed of the piezo and how fast can the objective be translated axially. This can be related to the resonant frequency of the combined piezo-objective system, and the maximum forces and accelerations facilitated in the system.
  • the fluorescence collection efficiency can scale as— ⁇ , with M being the objective magnification.
  • the "throw" - the distance with which piezo can travel, can also be limited, with the current state-of-the-art systems offering 400 ⁇ of total travel, although most systems have significantly less.
  • the settle time can also lead to lower duty cycles as imaging cannot take place during objective settling.
  • the exemplary SLM-based system/apparatus has significantly greater axial range, and couples no vibrations into the sample. It can be possible to currently scan at least three planes simultaneously at 10 Hz using traditional galvanometers, with over 500 ⁇ total separation between the outer planes (see, e.g., Figures 10A-10O), and the speed could be immediately tripled by installing a resonant galvanometer on the exemplary system.
  • change to a different axial plane can be performed very rapidly, in approzimately3ms, which can minimize lost time, and can facilitate axial switching within a frame, with little data loss (See, e.g., Figures 11 A-l 1C).
  • Remote focusing has also been used for faster volumetric imaging, either with the use of a secondary objective and movable mirror (see, e.g., References 9 and 10), or with
  • electrotunable lenses See, e.g., Reference 23. While both have higher performance than piezo mounted objectives, neither has yet demonstrated the ability to facilitate in-vivo functional imaging at the axial span illustrated herein.
  • Remote focusing with movable mirrors can scan with minimal aberration of the PSF, but for cell targeted imaging, defocusing induced aberrations may not be significant, neither in the exemplary procedure, nor for electrotunable lenses.
  • remote focusing with a mirror can offer better optical performance, but benefits from careful alignment and engineering, and may not be beneficial for somatic calcium imaging.
  • the electrotunable lens represents perhaps the most cost effective solution for high performance fast focusing, and can be inserted directly behind the objective.
  • AOD acousto-optic deflector
  • Point targeting systems need to densely target at a few ROIs in the sample to create a fiducial that can be used determine the magnitude and direction of motion. These fiducials need to be consistently visible, for closed loop correction.
  • a bigger limitation of all of these serially scanned systems can be that all can be nearing the fundamental limits of their speed, and further increases in performance can be likely only possible with multibeam systems.
  • the "speed- limit" can be given by the speed of sound in the material
  • a more practical limit can be set by the dwell time used per target to acquire signals with sufficient S R.
  • Single beam large FOV galvanometric scanning systems suffer from this as well, if one maintains high frame rates, the linear speed of travel of the excitation spot across the FOV can be high, resulting in low dwell times per unit distance on the sample. To maintain SNR at useful levels, finite dwell times can be beneficial. As fluorophore saturation ultimately can limit the maximum emission rate regardless of excitation intensity (e.g., increases in intensity simply cause photodamage, bleaching, and reduced spatial resolution).
  • the current "best" functional indicators like
  • GCaMP6 have high two-photon cross sections (see, e.g., Reference 14), and excitation saturation can be easily reached with modern laser systems. This can affect all single beam systems that use fluorescence as the optical readout.
  • An exemplary procedure to increase performance, and take advantage of available power, while maintaining sufficient dwell times for high sensitivity, can be to have multiple beams targeting the sample. Spatially multiplexed strategies have been used before (see, e.g., References 6, 8, 21, 32, and 37), but with very limited success for imaging neuronal activity in scattering samples.
  • Imaging in scattering samples can produce fluorescence, which can scatter extensively, limiting the ability to "assign" each fluorescence photon to its source (See, e.g., References 4, 3 and 32). Under these conditions, a more successful approach has been to time multiplex each separate excitation beam (See, e.g., References 4, 15, 19 and 20).
  • the standard lasers used in two-photon imaging operate at approximately 80 MHz, with the individual pulses separated by 12 ns, and many standard fluorophores used in biological imaging have fluorescence lifetimes of less than or equal to 3ns.
  • each single laser pulse can produce a short burst of fluorescence that can extend strongly for only 25% of the "dark time", rendering the system's duty cycle low.
  • the repetition rate can be effectively increased by a factor of 4, to approximately 320 MHz. with approximately 3 ns between pulses.
  • each beam can be adjusted to a different area and depth, and can be used for multilayer (see, e.g., Reference 15) or multi-area imaging. (See, e.g., Reference 58).
  • the exemplary system can exceed the demonstrated FOV and axial range of published implementation of temporally multiplexed beams, without customized electronics for time-stamped photon counting, and with significantly greater flexibility and speed in choosing targeted depths.
  • Other multibeam or temporally multiplexed strategies to be can be complementary methods that can be leveraged to further increase the exemplary system performance.
  • the performance of the exemplary system can depend on the SLM, a 512x512 pixel phase only device that performs beam-splitting by imparting phase modulation on the incoming laser pulse, and uses diffraction to redirect the beams to their targeted site.
  • the efficiency of any diffractive process can be affected by many things (e.g., the fill factor and the effective number of phase levels per "feature").
  • the controllable pixel fill factor of the exemplary SLM can be 82%, and thus, even with "perfect" holograms, the maximum power throughput can be reduced compared to a simple mirror. Taken together, these factors can lead to an effective power throughput on the SLM module ranging from approximately82% to approximately 40%, reducing the total light available for imaging.
  • the efficiency has not yet been a limitation for multiplane imaging, but it can be for imaging multiple planes in L5 and beyond. Compared to a conventional system with a piezo mounted objective, this can seem like a significant loss, however, it can be far less of a penalty than for multi-AOD systems, which give less than 20% total power throughput.
  • any diffractive device can be inherently chromatic (e.g., the deflection can depend on the wavelength of the light, laser sources for multi-photon imaging may not be strictly monochromatic (e.g., the ultrafast laser pulse has a characteristic bandwidth, with a minimal range dependent on the shape of the pulse.
  • the exemplary system e.g., Coherent Mira-HP
  • this can correspond to a FWHM of 6.5 nm at 940 nm for the approximately 140 fs pulses from the laser.
  • the spectral FWHM colors to be deflected ⁇ 1 ⁇ from the target position.
  • the effect of chromatic dispersion can be markedly less.
  • the dependence of focal length with wavelength can be /( ⁇ ) « fo ⁇ (j-), which can correspond to an axial shift between the FWHM points of approximately 1.1 ⁇ for a -200 ⁇ displacement, and approximately 1.6 ⁇ for a +300 ⁇ displacement.
  • the objective back aperture can be filled to have an NA of 0.45, and with these settings, the FWHM of the nominal monochromatic PSF can be approximately 6.5 ⁇ , so the chromatic axial displacement may only be a small fraction of the PSF, for the entire span of axial separations demonstrated here.
  • the axial range with high performance can be correspondingly smaller.
  • the relative displacement it produces, and the resultant axial PSF can, both scale as NA 2 .
  • NA 2 For example, at an excitation NA of 0.9, only 120 ⁇ of axial separation can be possible with the same relative performance.
  • fast multiplane calcium imaging with single cell resolution the performance can be excellent over 500 ⁇ .
  • the efficiency of this procedure can be improved by better SLMs (e.g., devices that have higher fill factors, increased pixel number and increased phase modulation. Increased pixel counts can increase the number of levels available across any feature, and can increase the diffraction efficiency. For nearly all applications, once the pixilation facilitates greater than or equal to 8 levels across a ramped 0 to 2 ⁇ transition, the efficiency can be very high. For arbitrary patterns of excitation, it may not be possible to predict how many overall pixels can be needed on the device to achieve this, but for the simple lateral or axial deflections, this can correspond to SLMs with approximately 1500 pixels across, which can keep the overall module power efficiency above 80% for the current fill factor, and higher with improved devices.
  • SLMs e.g., devices that have higher fill factors, increased pixel number and increased phase modulation.
  • Increased pixel counts can increase the number of levels available across any feature, and can increase the diffraction efficiency.
  • the efficiency can be very high.
  • SLMs e.
  • More pixels can also increase the possible range of deflections; however, this may not extend the range without better compensation of chromatic dispersion, which can be one reason why existing, higher pixel devices, may not be used.
  • the total size of the device may not be critical, and the pixel density may not need to be increased; just the total number of pixels.
  • Large pixels have an added benefit of increased power handling, and generally higher fill factors, because the electronics and insulating barriers between neighboring pixels can be somewhat fixed in size, whereas active areas may not be. Larger pixels can result in larger phase features, which can reduce the maximum angles of deflection, but this can be easily compensated with altering the magnification of the post-SLM telescope system, which can be beneficial to properly map the SLM to the back focal plane of the objective.
  • SLMs can be a natural element for corrections of both system, and sample aberrations ⁇ See, e.g., Reference 30).
  • Exemplary Volumetric Imaging The overall speed of multiplane imaging can be increased with additional exemplary procedures. Because of the systems sensitivity, activity with high S R can be detected. As such, the nominal dwell time per pixel can be reduced, and enough photons for effective detection can still be collected. By transitioning the exemplary microscope from conventional galvanometers to resonant galvanometers, the imaging can be sped up by at least a factor of three. Combined with the fast switching time of the exemplary device, large volumes of neural tissue can be imaged at high speed. The exemplary strategy can be to rapidly interleave multiplane images in successive scans to generate a complete picture of neural activity.
  • the exemplary SLM was chosen specifically because it can be controlled via low latency PCIe bus transfers, increasing speed, and giving deterministic latency for pattern changes.
  • This device was optimized for fast hologram transitions by increasing the backplane drive voltage, utilizing more than 2 ⁇ phase stroke, and the computation of optimized
  • Figure 1 IB shows a graph that illustrates the performance.
  • Two sets of planes in 3 ms can be fully transitioned between, with usable fluorescence produced in the new set of planes in 2 ms.
  • Imaging at 30 Hz, with three sets of 3-plane images, can be performed.
  • the fast transition times can ensure interframe switching while maximizing duty cycle. This can facilitate nine axial sections, with an overall volume rate of 10 Hz, over large regions of the cortex, and significant depth. ⁇ See, e.g., Figure 11C).
  • the imaged volume can also be increased by scaling up the number of independent beams entering the SLM modules, and incorporating various spatial and temporal multiplexing strategies ⁇ See, e.g., References 15 and 58). In these cases, the addressable volume or speed increase can scale linearly with the number of beams entering the SLM module(s).
  • the number of areas simultaneously illuminated can be deterministically controlled such that there can be direct control over the effective number of sources, in contrast to alternate extended two-photon approaches, like Bessel beam scanning ⁇ see, e.g., References 11 and 59), where the sample alone controls the complexity of the signal.
  • GCaMP6 has very low basal fluorescence, which may increase only with neuronal firing (e.g., the dark of space, punctuated by stars of activity). This has advantages for the signal unmixing and SNR, as background fluorescence can be minimal. As development continues on these and other probes, more sites can be targeted simultaneously.
  • the exemplary procedure functions well for imaging dendrites, or dendrites and soma ⁇ see, e.g., Reference 46) for dendritic source separation with CNMF.
  • the extended axial range of the exemplary procedure can facilitate exploring L5 soma and their apical tufts simultaneously, and can give direct insight into the role of dendritic spikes and computation in neuronal output. ⁇ See, e.g., References 35 and 55).
  • the SLM-based multiplane imaging can be a powerful procedure for addressing these and other questions that can benefit from high speed volumetric imaging with clear cellular resolution.
  • the exemplary system can be flexible, easily configurable and compatible with most existing two photon microscopes, and can provide new insights into cortical computation and function across multiple layers.
  • Virus AAVlsynGCaMP6f (see, e.g., Reference 14) was injected to both layer 2/3 and layer 5 of the left VI of the mouse cortex, 4-5 weeks prior to the craniotomy surgery.
  • the virus was front-loaded into the injection pipette and injected at a rate of 80 nl/min.
  • the injection sites were at 2.5 mm lateral and 0.3 mm anterior from the lambda, putative monocular region at the left hemisphere. Injections were made at two different depths from the cortical surface, layer 2/3 at 200 ⁇ -250 ⁇ and layer 5 at 400 ⁇ -500 ⁇ respectively.
  • mice were anesthetized with isoflurane (e.g., 2% by volume, in air for induction and 1% during surgery).
  • isoflurane e.g., 2% by volume, in air for induction and 1% during surgery.
  • dexamethasone sodium phosphate e.g., 2 mg per kg of body weight; to prevent cerebral edema
  • bupivacaine e.g., 5 mg/ml
  • enrofloxacin e.g., 4.47 mg per kg
  • an anti-inflammatory, carprofen e.g., 5 mg per kg
  • a circular craniotomy (e.g., 2 mm in diameter) was made above the injection site using a dental drill and the dura mater was removed (See, e.g., Reference 36). 1.5% agarose was placed over the craniotomy and a 3-mm circular glass coverslip (e.g., Warner instruments) was placed and sealed using a cyanoacrylate adhesive.
  • a titanium head plate with a 4 mm by 3.5 mm imaging well was attached to the skull using dental cement. The imaging experiments were performed 1- 14 days after the chronic window implantation. During the imaging, the mouse was awake and moved freely on a circular treadmill with its head fixed. Over 40 total imaging sessions across 17 mice were performed.
  • the shrimp used in the structural imaging were artemia nauplii (e.g., Brine Shrimp Direct) in hydra medium (e.g., with composition of ImM CaCl 2 *H 2 0, O. lmM MgCl 2 *6H 2 0, 30 ⁇ KN0 3 , 0.5mM NaHC0 3 , 80 ⁇ MgS0 4 ).
  • hydra medium e.g., with composition of ImM CaCl 2 *H 2 0, O. lmM MgCl 2 *6H 2 0, 30 ⁇ KN0 3 , 0.5mM NaHC0 3 , 80 ⁇ MgS0 4 .
  • the oxygen concentration in the medium was reduced by limiting perfusion, which can lead to reduced motion of the shrimp.
  • the setup of the exemplary two-photon SLM laser scanning microscope is illustrated in Figure 1 A.
  • the laser source can be a pulsed Ti: Sapphire laser (e.g., Coherent Mira HP) which can be tuned to 940 nm with a maximum output power of approximately 1.4 W (e.g., approximately 140 fs pulse width, 80 MHz repetition rate).
  • the output laser beam can be directed through a Pockets cell (e.g., ConOptics EO350-160-BK) to control the intensity on the sample.
  • a broadband ⁇ /2 waveplate (e.g., Thorlabs AHWP05M-980) was located after the EO modulator to rotate the laser polarization so that it can be parallel with the active axis of the spatial light modulator (e.g., Meadowlark Optics, HSP512-1064, 7.68 ⁇ 7.68 mm 2 active area, 512 x 512 pixels) located further downstream.
  • the light incident angle to the SLM can be approximately 3.5o.
  • XLPlan N objective can be used for the imaging. Other objectives also perform well in the exemplary system.
  • the SLM surface can be imaged afocally to the back pupil of the objective.
  • the fluorescent signal from the sample can be detected with a photo-multiplier tube ("PMT", Hamamatsu H7422P-40) located above the microscope, and followed by a low noise amplifier (Stanford Research Systems SR570).
  • Scanlmage 3.8 ⁇ see, e.g., Reference 47) can be used to control the galvanometer mirrors, digitize and store the signal from the amplifier of the PMT.
  • the line scanning can be bidirectional with a single line scan rate of 2 kHz. For a 256 x 200 pixel imaging, the frame rate can be 10 fps.
  • Custom software using MATLAB was developed to load and control the phase hologram pattern on the SLM through a PCIe interface (Meadowlark Optics).
  • the SLM was calibrated, and a lookup table that converts the pixel values to voltage driving the liquid crystal of the SLM was generated.
  • the SLM outputs approximately 80 effective phase levels over a 2 ⁇ phase range, with a relative uniform phase level spacing. ⁇ See, e.g., graph shown in Figure 7A).
  • phase hologram pattern on the SLM ⁇ , v can be, for example expressed in the following:
  • ⁇ ( ⁇ , ⁇ ) phase ⁇ [l 1 A i e 2 ⁇ i XiU+ y iU+ [ z ° (u ' v)c ° (Zi)+z ° (u ' v)c ° (Zi)+z ⁇ (Ml)
  • Ai can be the electrical field weighting factor for the individual beamlet.
  • ⁇ (u, v) and Cm(Zi) can be the Zernike polynomials and Zernike coefficients, respectively, which can fulfill the defocusing functionality and can compensate the first-order and second-order spherical aberration due to defocusing.
  • the expressions of Zm (u, v) and Cm(zi ) are shown below. (See, e.g., Reference 5).
  • a 2D coordinate calibration between the SLM phase hologram, and thus the projected beamlet pattern at the sample through Eq. M and the PMT image can be carried out on a pollen grain slide.
  • an affine transformation can be extracted to map the coordinates between the PMT image and the image plane at the sample, and thus the SLM phase hologram through Eq. Ml .
  • the defocusing length set in the SLM phase hologram can be matched with the actual defocusing length by adjusting the apparent "effective N.A.” the Zernicke coefficients, after calibration following the procedure described in (See, e.g., Reference 49). This can be done mainly for convenience, and it can be noted that this can change very little over the full axial range of the SLM (e.g., a range of 0.43-0.48).
  • the field weighting factor Ai in Eq. Ml can determine the power ratio, of different imaging planes. It can be important to note that this coefficient may not be simply the desired power ratio rather it can affect the interference between the different fields, which in turn can depend on the relative phase displacements of the beams. Operationally, either the parameter can be adjusted empirically to achieve similar fluorescent signals from different imaging planes, or alternately, the expected power ratio from first principles can be calculated, considering the depths of each plane, nominal scattering length of light in the tissue, and perform numerical beam propagation of the electric field ⁇ See, e.g., Reference 54).
  • Visual stimuli were generated using MATLAB (e.g., The MathWorks, Natick, MA) and the Psychophysics Toolbox ⁇ see, e.g., Reference 12) and displayed on a liquid crystal display (“LCD") monitor (e.g., 19-inch, 60-Hz refresh rate) position 28 cm from the right eye, at approximately 45o to the long axis of the animal.
  • LCD liquid crystal display
  • Each visual stimuli session consisted of 8 different trials, each trial with a 3 s drifting square gratings (e.g., 100% contrast, 0.035 cycles per degree, two cycles per second) display, followed by 5 s of mean luminescence gray screen. 8 drifting directions (e.g., separated by 45 degrees) were presented in random order in the 8 trials in each session.
  • 17 sessions were recorded continuously (e.g., 1088 s).
  • a silicon photodetector e.g., Thorlabs DET 36 A
  • the detector output was recorded simultaneously with the imaging data at Scanlmage 3.8.
  • the neuronal responses to the visual stimulation can then be extracted and analyzed in the subsequent image processing.
  • the raw images can first be processed to correct brain motion artifacts using an exemplary pyramid procedure ⁇ see, e.g., Reference 60), and then analyzed using a constrained non-negative matrix factorization procedure, coded in MATLAB (e.g., The MathWorks, Natick, MA).
  • MATLAB e.g., The MathWorks, Natick, MA.
  • the signals extracted from CNMF can be compared with the NOL signals in two aspects: similarity and signal-to-noise ratio ("SNR").
  • SNR can be calculated based on the signal's power spectral density.
  • ICA was also performed to analyze the data, with the software written in Matlab ⁇ See, e.g., Reference 39).
  • the motion-corrected image recording stack can first be normalized, followed by principal component analysis ("PCA") for dimensionality reduction and noise removal.
  • PCA principal component analysis
  • ICA can then be applied to extract the spatio-temporal information of each
  • independent sources e.g., ROI
  • the total number of events can be counted during the visual stimulation period in each session for all 8 different grating angles. These event numbers can be into a vector space ⁇ See, e.g., Reference 38). The direction and magnitude of their vector sum can represent the orientation selectivity and the orientation index.
  • N v i SU aisession visual stimulation sessions N V i SU aisession vectors can be obtained. Hotelling's T 2 -test can be used to calculate whether these vectors can be significantly different from 0, for example, whether the ROI has a strong orientation selectivity.
  • ⁇ ⁇ d ⁇ sin(0) (SI)
  • SI the maximum SLM induced deflection angle
  • the efficiency of redirection can depend on the number of phase levels available on the SLM.
  • the exemplary device was measured, and approximately 80 usable phase levels between 0 and 2 ⁇ were found. (See, e.g., Figure 7A). With the simplifying assumption that these levels can be uniformly spaced between 0 and 2 ⁇ , the first order diffractive efficiency of such a device can be given by for example:
  • N can be the total number of levels.
  • N can be the total number of levels.
  • This can imply very high efficiencies for holograms with large features (e.g., read spanning many pixels), which can result in small positional displacements in the sample (e.g., Fourier) plane.
  • the beneficial phase features can get smaller and smaller, and the finite sampling from the pixelated SLM can reduce the available phase levels.
  • there can be aliasing, which can strongly degrade the performance; (i)that limit can be avoided for all holograms used.
  • Four pixels can be considered across a ⁇ 2 ⁇ ramp to be the limit of efficient power redirection. The deflection vs.
  • the pattern can have only two phase levels, which can lower the maximum diffraction efficiency, and additionally, can result in a symmetric phase pattern.
  • the phase "grating" may not be blazed, but instead can split the beam equally to the +1 and -1 orders; there can be two spots, mirrored across the nominal center of the FOV. This may not be a problem for dual lateral region imaging, where such a split can be in fact advantageous.
  • the desired effect can be to have a single targeted point on the sample, it can be best to restrict the lateral targeting to regions that can be address by phase gratings that span at least four pixels (e.g., levels). With four pixels blaze and two photon excitation, the excitation ratio of the +1 to -1 order can be ⁇ 20: 1 ; for five pixels, approximately 50: 1 ; and for eight pixels, approximately 350: 1.
  • the SLM can be used to impart a spherical curvature to the wavefront that can shift the focus before or beyond the nominal focal plane.
  • only primary defocus can be considered here, neglecting higher order aberration terms.
  • the normalized pupil radius, p the Fresnel zone order, f, the focal length and ⁇ the wavelength.
  • Each successive zone can correspond to adding one wave of defocus at that point on the pupil, which can also correspond to a 2 ⁇ phase change.
  • the "strongest" lens allowed without aliasing on the exemplary actual device can correspond to a two pixel spacing on the outermost Fresnel zone.
  • the relative spacing between zones can be, for example:
  • the sign of f can be made positive or negative simply by reversing the phase.
  • this lens power can correspond to axial displacements of ⁇ 300 ⁇ under the objective. Any larger displacement can result in aliasing on the SLM, which can both reduce diffraction efficiency, as well as create secondary foci.
  • the image of the SLM on the back aperture can have higher pixel density.
  • the exemplary preference can be to have more pixels rather than smaller pixels (e.g., smaller pixels typically have lower fill factors, and larger pixels can have increased power handling, as the overall power density on the physical device can be decreased with increasing SLM area).
  • the fill factor can be important for two reasons. The first can be the overall power transmission to the diffracted beam, which can be directly proportional to the fill factor. The second reason can be more subtle.
  • SLMs can be diffractive optics, so they can be chromatic devices where beam redirection can depend on wavelength, (See, e.g., Equation (SI)).
  • any non-monochromatic source e.g. femtosecond pulsed laser
  • the focal length can change with the wavelength, and axial color can be similar, and proportional to 7-.
  • NA approximately 0.45
  • the FWHM of the axial two photon point spread function can be approximately e.g., References 64 and 69).
  • the axial color across the FWHM of the laser spectrum can be approximately 2 ⁇ , which represents a total shift of 25% of the axial PSF. This can have a negligible effect on the axial resolution, and power delivery for somatic imaging. Assuming the excitation NA can be changed, the relative effect of the axial color can be invariant with NA, though the absolute sizes of everything scale as NA 2 .
  • Figures 8A and 8B are shows ROI contour and calcium signals according to an exemplary embodiment of the present disclosure.
  • the ROI contour and calcium signal extracted by CNMF is the contour overlaid on the temporal standard deviation image of a single plane recording of mouse VI at depth of 500 ⁇ for cortical surface.
  • Traces 810 are the normalized AF/F traces with a scale bar of 50 ⁇ . Clean, cell like features can be selected, and the great majority of traces have signals with high SNR, and that appear like familiar Ca++ activity traces.
  • Figure 8B is similar to Figure 8A, with ROI contour and calcium signal extracted by ICA. The ROIs can be less cell- like, and a large number of extracted traces show poor SNR, with no features recognizable as typical activity traces.
  • the mouse brain can be highly scattering, and there can be an exponential loss of ballistic photons with increasing depth.
  • the scattering length can be approximately 185 ⁇ (See, e.g., References 27 and 70).
  • two- photon fluorescence signal can scale as the intensity squared. If it can be assumed that the expression of the indicator can be the same in the desired layers for imaging, the relative power needed to be delivered to each layer to give equivalent signal can be estimated as, for example:
  • this ratio can be as follows: n -2(500 - 170)
  • Figures 10A-10O show the Axial Three Plane in-vivo Functional Imaging of Mouse VI at Layer 2/3, 4 and 5, and Extended Simultaneous Imaging Depth over 500 ⁇ .
  • the top panels of Figures 1 OA- IOC show, temporal standard deviation images of the sequential single plane recording of mouse VI at depth of 170 ⁇ (e.g., layer 2/3), 350 ⁇ (e.g., layer 4) and 500 ⁇ (e.g., layer 5) from a pial surface.
  • the bottom panels of Figures lOA-lOC show ROI contours overlaid on the top panel. The images are false-colored. Scale bar, 50 ⁇ .
  • Figure 10D shows an image of the arithmetic sum of Figures 1 OA- IOC.
  • the top panel of Figure 10E shows the temporal standard deviation imaging of the simultaneous three-plane recording of the planes shown in Figures 1 OA- IOC.
  • the bottom panel of Figure 10E shows an exemplary image of overlaid ROI contours from the three plane.
  • Figure 10F shows a trace diagram with represented extracted AF/F traces of the selected ROIs from the three planes (e.g., trace 1005, layer 2/3, 15 ROIs out of 58; trace 1010, layer 4, 15 ROIs out of 65; trace 1015, layer 5, 15 ROIs out of 95), from the sequential single plane recording.
  • Figure 10G shows an exemplary trace diagram of extracted AF/F traces of the same ROIs shown in Figure 10F, from the simultaneous three-plane recording.
  • Figure 10H shows an exemplary diagram of the source separation of the fluorescent signal from spatially overlapped ROIs in the three plane imaging. The contours of the overlapped ROIs are plotted with their ROI ID. The calcium traces are normalized AF/F traces. For each ROI, the signal is extracted using NOL shown by element 1020, and CNMF by element 1025. The corresponding ROI contours results from these methods are shown next to their AF/F traces. Using the ROI contour in the CNMF but with uniformed pixel weighting and without unmixing treatment, the extracted AF/F trace 1030 is plotted, superimposed onto the traces extracted from CNMF.
  • Figure 101 shows an exemplary diagram of the simultaneous three-plane imaging extending the imaging depth over 500 ⁇ .
  • Figure 10J shows an exemplary overlaid temporal standard deviation image of the sequential single plane recording of mouse VI at depth of 130 ⁇ (e.g., element 1035), 430 ⁇ (e.g., element 1040) and 640 ⁇ (e.g., element 1045) from pial surface of a mouse VI with a scale bar of 50 ⁇ .
  • Figure 10K shows an exemplary temporal standard deviation imaging of the simultaneous three-plane recording of the three planes.
  • Figure 10L shows an exemplary image of the overlaid ROI contours from the three plane on Figure 10 J, while Figure 10M shows the extracted ROI contours.
  • Figure ION shows represented extracted AF/F traces of the selected ROIs from the three planes (e.g., trace 1045, layer 2/3; trace 1050, layer 4; trace 1055, layer 5), from the sequential single plane recording.
  • Figure 10O shows the extracted AF/F traces of the same ROIs shown in Figure 10N, from the simultaneous three-plane recording.
  • Figure 11 A shows an exemplary diagram of SLM switching between two sets of dual plane imaging on mouse VI .
  • State 1 is the dual plane for depth of 210 ⁇ and 330 ⁇ from cortical surface
  • state 2 is the dual plane for depth of 110 ⁇ and 270 ⁇ from pial surface.
  • Imaging frame rate is 10 fps.
  • the SLM switching happens at the middle and at the end of each frame.
  • the zoom-in-view of the switching region shows that the switching time between the two state is less than 2.5 ms.
  • Figure 1 IB shows a graph of the switching time between State 1 and State 2, measure from fluorescent signal emitted from Rhodamine 6G.
  • the switching time between different states is less than 3 ms.
  • the black indicator 1105 indicates when the switching starts.
  • Figure l lC shows a diagram of a volumetric imaging procedure using time-multiplexed three-plane imaging.
  • the normal frame rate is 30 fps.
  • the volumetric imaging rate is 10 vol/s.
  • FIG. 13 shows an exemplary method 1200 according to an exemplary embodiment of the present disclosure.
  • a radiation can be generated using, for example, a laser source.
  • the generated radiation can be provided to a SLM, which can split the radiation into a plurality of radiation beamlets at procedure 1215.
  • the SLM can independently dynamically control the radiation beamlets and provide them to a galvanometer.
  • the beamlets can be directed to the sample using the galvanometer (e.g., to different areas and/or to different planes), which can be used to sweep the beamlets across the sample at or 1230.
  • a resultant radiation can be received from the sample that can be based on the beamlets.
  • a plurality of images can be generated based on the resultant radiation.
  • the images can be interleaved, and one or more multiplane images can be generated at procedure 1250.
  • brain motion artifacts can be corrected for in the one or more multiplane images. Additionally, or in the alternative, brain notion artifacts can be corrected for in the plurality of images before the one or more multiplane images are generated.
  • Figure 13 shows a block diagram of an exemplary embodiment of a system according to the present disclosure.
  • exemplary procedures in accordance with the present disclosure described herein can be performed by a processing arrangement and/or a computing arrangement 1302.
  • Such processing/computing arrangement 1302 can be, for example entirely or a part of, or include, but not limited to, a computer/processor 1304 that can include, for example one or more microprocessors, and use instructions stored on a computer-accessible medium (e.g., RAM, ROM, hard drive, or other storage device).
  • a computer-accessible medium e.g., RAM, ROM, hard drive, or other storage device.
  • a computer-accessible medium 1306 e.g., as described herein above, a storage device such as a hard disk, floppy disk, memory stick, CD- ROM, RAM, ROM, etc., or a collection thereof
  • the computer-accessible medium 1306 can contain executable instructions 1308 thereon.
  • a storage arrangement 1310 can be provided separately from the computer-accessible medium 1306, which can provide the instructions to the processing arrangement 1302 so as to configure the processing arrangement to execute certain exemplary procedures, processes and methods, as described herein above, for example.
  • the exemplary processing arrangement 1302 can be provided with or include an input/output arrangement 1314, which can include, for example a wired network, a wireless network, the internet, an intranet, a data collection probe, a sensor, etc.
  • the exemplary processing arrangement 1302 can be in communication with an exemplary display arrangement 1312, which, according to certain exemplary embodiments of the present disclosure, can be a touch-screen configured for inputting information to the processing arrangement in addition to outputting information from the processing arrangement, for example.
  • the exemplary display 1312 and/or a storage arrangement 1310 can be used to display and/or store data in a user-accessible format and/or user-readable format.
  • MMM Multifocal multiphoton microscopy
  • PICCS Prior image constrained compressed sensing

Abstract

An exemplary device can be provided which can include, for example, a radiation source(s) configured to generate a first radiation(s), a spatial light modulator (SLM) arrangement(s) configured to receive the first radiation(s) and generate a second radiation(s) based on the first radiation(s), and a galvanometer(s) configured to receive the second radiation(s), generate a third radiation(s) based on the second radiation(s), and provide the third radiation(s) to a sample(s).

Description

SYSTEM, METHOD AND COMPUTER-ACCESSIBLE MEDIUM FOR MULTI-PLANE
IMAGING OF NEURAL CIRCUITS
CROSS-REFERENCE TO RELATED APPLICATIONS)
[0001] This application relates to and claims priority from U.S. Patent Application No.
62/187,595, filed on July 1, 2015, the entire disclosure of which is incorporated herein by Reference.
FIELD OF THE DISCLOSURE
[0002] The present disclosure relates generally to multi-plane imaging, and more specifically, to exemplary embodiments of an exemplary system, method and computer-accessible medium for, e.g., simultaneous multi-plane imaging of neural circuits.
BACKGROUND INFORMATION
[0003] A coherent activity of individual neurons, firing in precise spatiotemporal patterns, can likely be the underlying basis of thought and action in the brain. Optical imaging methods aim to capture this activity, with recent progress now facilitating the functional imaging of nearly the entire brain of an intact transparent organism, the zebra fish, with cellular resolution. {See, e.g., Reference 1). In scattering tissue, where nonlinear microscopy can be beneficial {see, e.g., References 23, 63, and 69), progress toward imaging large pools of neurons has been slower. But in nearly all existing two-photon microscopes, a single beam can be serially scanned in a continuous trajectory across the sample with galvanometric mirrors, in a raster patterns or with a specified trajectory that intersects targets of interest along the path. This means that the imaging can be serial and thus slow.
[0004] Since the inception of two-photon microscopy, there have been large efforts to increase the speed and extent of imaging. Parallelized multifocal approaches have been developed {see, e.g., References 8 and 56 ), as well as inertia-free scanning using acousto-optic deflectors ("AODs") {see, e.g., References 23, 31, and 50), or scanless approaches utilizing spatial light modulators ("SLMs") {see, e.g., References 16, 41 and 48), each with its own strengths and weaknesses. Despite the tremendous improvements in imaging modalities, the "view" can still be limited, whether by the fundamental technology, the expense or the complexity. A difficulty in imaging can be linked to expanding the volumetric extent of imaging, while maintaining high temporal resolution and high sensitivity. {See, e.g., References 2 and 3). This can generally be linked to the inverse relationship between volume scanned, and the signal collected per voxel, at a fixed resolution.
[0005] Thus, it may be beneficial to provide an exemplary system, method and computer- accessible medium which can overcome at least some of the deficiencies described herein above.
SUMMARY OF EXEMPLARY EMBODIMENTS
[0006] To that end, in order to overcome some of the deficiencies presented herein above, an exemplary device can be provided which can include, for example, a radiation source(s) configured to generate a first radiation(s), a spatial light modulator (SLM) arrangement(s) configured to receive the first radiation(s) and generate a second radiation(s) based on the first radiation(s), and a galvanometer(s) configured to receive the second radiation(s), generate a third radiation(s) based on the second radiation(s), and provide the third radiation(s) to a sample(s).
[0007] In some exemplary embodiments of the present disclosure, the SLM arrangement(s) can include, for example, a SLM and a pre-SLM afocal telescope configured to resize the first radiation(s) to match an area of the SLM. According to particular exemplary embodiments of the present disclosure, the SLM arrangement(s) can further include a plurality of folding mirrors configured to redirect the first radiation(s) to the pre-SLM, and a post SLM afocal telescope configured to resize the second radiation(s) to match a size of an acceptance aperture of the galvanometer(s). The SLM arrangement(s) can also further include a broadband waveplate(s) located between the pre-SLM afocal telescope and the SLM. The broadband waveplate(s) can be configured to rotate a polarization of the first radiation(s) to cause the radiation(s) to be parallel with an active axis of the SLM.
[0008] In certain exemplary embodiments of the present disclosure, the SLM arrangement(s) can be configured to split the first radiation(s) into the radiation beamlets which are the second radiation(s). The SLM arrangement(s) can be further configured to independently dynamically control each of the radiation beamlets. The SLM arrangement(s) can split the first radiation(s) into the radiation beamlets by imprinting a phase profile across the first radiation(s). The galvanometer(s) can be further configured to direct each of the radiation beamlets to a different area of the sample(s). The galvanometer(s) can direct each of the radiation beamlets to a different plane of the sample(s). The radiation source(s) can be a laser source(s). [0009] In some exemplary embodiments of the present disclosure, a computer processing arrangement can be configured to generate an image(s of the sample(s) based on a plurality of resultant radiations received from the sample(s) that can be based on the radiation beamlets. A first number of the resultant radiations can be based on a second number of the radiation beamlets. The second number of the radiation beamlets can be based on a third number of the planes of the sample(s). The computer processing arrangement can be further configured to generate a third number of images of the sample(s) based on the resultant radiations.
[0010] In certain exemplary embodiments of the present disclosure, the image(s) can include a plurality of images. The computer processing arrangement can be further configured to generate a multiplane image(s) based on the images. The multiplane image(s) can be generated by interleaving the images into the multiplane image(s). The computer processing arrangement can be further configured to correct brain motion artifacts in the images based on a pyramid procedure.
[0011] According to some exemplary embodiments of the present disclosure, a pocket cell(s) can be located between the radiation source(s) and the SLM arrangement(s), which can be configured to modulate an intensity of the first radiation(s). A computer processing arrangement can be provided, which can be configured to generate an image(s) of the sample(s) based on a fourth radiation(s) received from the sample(s) that can be based on the third radiation(s).
[0012] According to a further exemplary embodiment of the present disclosure, an exemplary method can include, for example, generating a radiation(s), providing the radiation(s) to a spatial light modulator (SLM) arrangement(s), splitting the radiation(s) into a plurality of radiation beamlets using the SLM arrangement(s), and directing the radiation beamlets to a sample(s) using a galvanometer(s). A computer hardware arrangement can be used to generate an image(s) of the sample(s) based on a resultant radiation received from the sample(s) that can be based on the radi ati on b eaml ets .
[0013] In some exemplary embodiments of the present disclosure, the radiation(s) can be generated using a laser(s). Each of the radiation beamlets can be independently dynamically controlled using the SLM arrangement(s). The SLM arrangement(s) can split the radiation(s) into the plurality of radiation beamlets by imprinting a phase profile across the radiation(s). The SLM arrangement(s) can include a SLM(s). Each of the radiation beamlets can be directed to a different area of the sample(s). Each of the radiation beamlets can be swept across the respective different area of the sample(s) using the galvanometer(s).
[0014] In certain exemplary embodiments of the present disclosure, each of the radiation beamlets can be directed to a different plane of the sample(s). A plurality of resultant radiations can be received from the sample(s) that can be based on the radiation beamlets. A first number of the resultant radiations can be based on a second number of the radiation beamlets. The second number of the radiation beamlets can be based on a third number of the planes of the sample(s). A third number of images of the sample(s) can be generated based on the resultant radiations. A plurality of images can be generated based on the resultant radiations.
[0015] In some exemplary embodiments of the present disclosure, a multiplane image(s) can be generated based on the images. The multiplane images can be generated by interleaving the images into the multiplane image(s). Brain motion artifacts can be corrected in the images based on a pyramid procedure.
[0016] These and other objects, features and advantages of the exemplary embodiments of the present disclosure will become apparent upon reading the following detailed description of the exemplary embodiments of the present disclosure, when taken in conjunction with the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Further objects, features and advantages of the present disclosure will become apparent from the following detailed description taken in conjunction with the accompanying Figures showing illustrative embodiments of the present disclosure, in which:
[0018] Figure 1 A is an exemplary diagram of an exemplary SLM Two-photon Microscope according to an exemplary embodiment of the present disclosure;
[0019] Figures IB and 1C are exemplary diagrams of axial and lateral dual plane imaging according to an exemplary embodiment of the present disclosure;
[0020] Figure ID is a set of exemplary images of two-photon structural imaging of a shrimp (e.g., artemia nauplii) at different depths of the sample according to an exemplary embodiment of the present disclosure;
[0021] Figure IE is a set of exemplary images of software based SLM focusing of the same shrimp as in Figure ID according to an exemplary embodiment of the present disclosure; [0022] Figure IF is an exemplary image of the sum of all the images at the seven planes shown in Figure ID according to an exemplary embodiment of the present disclosure;
[0023] Figure 1G is an exemplary image of the sum of all the images at the seven planes shown in Figure IE according to an exemplary embodiment of the present disclosure;
[0024] Figure 1H is an exemplary image of seven-axial-plane imaging of the same shrimp as in Figures 1D-1G according to an exemplary embodiment of the present disclosure;
[0025] Figure II is an exemplary image of seven-axial-plane imaging using the SLM to increase the illumination intensity only for the 50 μιη plane according to an exemplary embodiment of the present disclosure;
[0026] Figure 2A is an exemplary diagram of the exemplary in-vivo experiment imaging the VI of the mouse according to an exemplary embodiment of the present disclosure;
[0027] Figures 2B and 2C are exemplary images of the temporal standard deviation image of the sequential single plane recording of mouse VI at a depth of about 280 μιη from the pial surface according to an exemplary embodiment of the present disclosure;
[0028] Figure 2D is an exemplary image of some of the illustrations from Figures 2B and 2C according to an exemplary embodiment of the present disclosure;
[0029] Figure 2E is an exemplary image of the temporal standard deviation image of the simultaneous dual plane recording of the two fields of view according to an exemplary embodiment of the present disclosure;
[0030] Figure 2F is an exemplary signal diagram of the extracted AF/F traces of the selected ROIs from the two field of views according to an exemplary embodiment of the present disclosure;
[0031] Figure 2G is an exemplary signal diagram of extracted AF/F traces of the same ROIs shown in Figure 2F from the simultaneous dual plane recording according to an exemplary embodiment of the present disclosure;
[0032] Figure 2H is an exemplary signal diagram of a zoomed in view of the normalized AF/F traces in the shaded area in Figure 2G according to an exemplary embodiment of the present disclosure;
[0033] Figures 3 A and 3B are exemplary images of the temporal standard deviation image of the sequential single plane recording of mouse VI at a depth of about 170 μπι (e.g., layer 2/3) and depth of about 500 μηι (e.g., layer 5) from the cortical surface according to an exemplary embodiment of the present disclosure;
[0034] Figure 3C is an exemplary image using the exemplary system according to an exemplary embodiment of the present disclosure;
[0035] Figure 3D is an exemplary image of the temporal standard deviation image of the simultaneous dual plane recording of the two planes shown in Figures 3 A and 3B according to an exemplary embodiment of the present disclosure;
[0036] Figure 3E is an exemplary signal diagram of the extracted AF/F traces of 20 ROIs out of 350 from the two planes from the sequential single plane recording according to an exemplary embodiment of the present disclosure;
[0037] Figure 3F is an exemplary signal diagram of extracted AF/F traces of the same ROIs shown in Figure 3E from the simultaneous dual plane recording according to an exemplary embodiment of the present disclosure;
[0038] Figures 3G-3I are zoomed views of the exemplary signal diagrams of AF/F traces in the shaded area in Figures 3E and 3F according to an exemplary embodiment of the present disclosure;
[0039] Figures 4A-4C are exemplary signal diagrams of the source separation of the fluorescent signal from spatially overlapped ROIs in the dual plane imaging shown in Figures 3 A-3I according to an exemplary embodiment of the present disclosure;
[0040] Figure 4D is an exemplary signal diagram of the ROI contour map showing the ROI locations in Figures 4A-4C according to an exemplary embodiment of the present disclosure;
[0041] Figure 4E is an exemplary chart illustrating the correlation coefficient between the AF/F extracted from MF and NOL for a total of 250 ROIs according to an exemplary embodiment of the present disclosure;
[0042] Figure 4F is an exemplary graph illustrating the signal-to-noise ratio between the AF/F extracted from NMF and the raw AF/F extracted from NOL for a total of 250 ROIs according to an exemplary embodiment of the present disclosure;
[0043] Figure 5A is a set of exemplary signal diagrams of normalized AF/F traces of selected 150 ROIs in the dual axial plane imaging in Figures 3A-3I according to an exemplary embodiment of the present disclosure; [0044] Figure 5B is an exemplary chart illustrating the inter-laminar correlation map of activity between neurons in L2/3 and L5 according to an exemplary embodiment of the present disclosure;
[0045] Figure 5C is an exemplary signal diagram of a pair of ROIs that show relatively high correlation (e.g., R=0.1709) according to an exemplary embodiment of the present disclosure;
[0046] Figure 5D is an exemplary chart illustrating the activity correlation coefficient extracted from Figure 5B versus the lateral distance of the corresponding pairs of ROIs according to an exemplary embodiment of the present disclosure;
[0047] Figure 6A is an exemplary signal diagram of normalized AF/F traces for selected ROIs with strong response to drifting grating visual stimulation, recorded with simultaneous dual plane imaging according to an exemplary embodiment of the present disclosure;
[0048] Figure 6B is an exemplary diagram and an exemplary chart illustrating of the response of the ROIs to the drifting grating in visual stimulation according to an exemplary embodiment of the present disclosure;
[0049] Figure 6C is an exemplary diagram and an exemplary chart illustrating the response of the ROIs to the drifting grating in visual stimulation for ROIs located at 450 μπι depth from cortical surface according to an exemplary embodiment of the present disclosure;
[0050] Figure 6D1 is an overlaid temporal standard deviation image of the sequential single plane recording of the 200 μπι plane and 450 μπι plane according to an exemplary embodiment of the present disclosure;
[0051] Figure 6D2 is an exemplary image of extracted ROI contours from the two planes with a Scale bar of 50 μπι according to an exemplary embodiment of the present disclosure;
[0052] Figure 6E is an exemplary diagram of the evoked responses of the ROIs with spatial lateral overlaps from the two planes according to an exemplary embodiment of the present disclosure;
[0053] Figure 7A is an exemplary graph illustrating the reflected phase of the SLM for a wavelength of 940 nm versus the applied driving pixel value to the SLM according to an exemplary embodiment of the present disclosure;
[0054] Figure 7B is an exemplary graph illustrating SLM two-photon fluorescence efficiency with different defocusing length, measured from the fluorescence emitted from Rhodamine 6G with two-photon excitation according to an exemplary embodiment of the present disclosure; [0055] Figure 7C is an exemplary graph illustrating SLM lateral deflection efficiency, measured from the optical power at the back aperture of the objective according to an exemplary embodiment of the present disclosure;
[0056] Figure 7D is an exemplary graph illustrating calculated SLM lateral deflection efficiency according to an exemplary embodiment of the present disclosure;
[0057] Figure 7E is an exemplary image of 12 spots generated by SLM simultaneously imaged on a CCD according to an exemplary embodiment of the present disclosure;
[0058] Figure 7F is an image of 12 spots generated by about SLM simultaneously imaged on CCD with power compensation for each spot for SLM lateral deflection efficiency according to an exemplary embodiment of the present disclosure;
[0059] Figures 8A and 8B are exemplary ROI contour and calcium signals according to an exemplary embodiment of the present disclosure;
[0060] Figure 9A is an exemplary chart illustrating the correlation coefficient between the AF/F extracted from C MF and NOL for a total of 250 ROIs according to an exemplary embodiment of the present disclosure;
[0061] Figures 9B-9G are exemplary charts illustrating ROIs with various correlation coefficient between CNMF and NOL according to an exemplary embodiment of the present disclosure;
[0062] Figures 10A-10H are exemplary images and signal charts of three plane imaging, on mouse VI at depth of 170 μπι, 350 μπι and 500 μπι from pial surface according to an exemplary embodiment of the present disclosure;
[0063] Figures 10I-10O are exemplary images and signal charts of three plane imaging, on VI at depth of 130 μπι, 430 μπι and 640 μπι from pial surface, extending a simultaneous imaging depth over 500 μπι according to an exemplary embodiment of the present disclosure;
[0064] Figure 11 A is a set of exemplary images of SLM switching between two sets of dual plane imaging on mouse VI according to an exemplary embodiment of the present disclosure;
[0065] Figure 1 IB is an exemplary chart illustrating switching time between State 1 and State 2, measured from fluorescent signal emitted from Rhodamine 6G according to an exemplary embodiment of the present disclosure; [0066] Figure 11C is an exemplary diagram of a volumetric imaging procedure using time- multiplexed three-plane imaging according to an exemplary embodiment of the present disclosure; and
[0067] Figure 12 is an illustration of an exemplary block diagram of an exemplary system in accordance with certain exemplary embodiments of the present disclosure.
[0068] Throughout the drawings, the same Reference numerals and characters, unless otherwise stated, are used to denote like features, elements, components or portions of the illustrated embodiments. Moreover, while the present disclosure will now be described in detail with Reference to the figures, it is done so in connection with the illustrative embodiments and is not limited by the particular embodiments illustrated in the figures and the appended claims.
DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS
Exemplary Results
[0069] An exemplary embodiment of the present disclosure can include, for example an exemplary SLM microscope coupled with a two-photon microscope, galvanometers and an SLM module. Figure 1 A shows a schematic diagram of the multi-plane imaging and the exemplary SLM two-photon microscope apparatus. The exemplary SLM module can include diverting the input path of the microscope, prior to the galvanometer mirrors 140, using retractable kinematic mirrors, onto a compact optical breadboard with the SLM 130 and associated exemplary components. Exemplary features of the SLM module can include folding mirrors for redirection, a pre-SLM afocal telescope 115 to resize the incoming beam to match the active area of the SLM, the SLM 130 and a post-SLM afocal telescope 135to resize the beam again to match the acceptance aperture of the galvanometers 140, and to fill the back focal plane of the objective 155 appropriately.
[0070] The exemplary SLM 130, post-SLM telescope 135 and galvanometers 140 can be spaced such that the SLM 130 can be conjugate to the galvanometers 140, and the microscope scan lens 145, and tube lens 150, can reimage this again to the back aperture of the microscope objective. This module can be coupled to 2P microscopes, and to Prairie/Bruker systems, with equal success, and similar performance. The SLM 130 can be used as a flexible, programmable beam splitter that can facilitate independent dynamic control of each generated beamlet, at high speed. The exemplary SLM 130 can perform this flexible beamsplitting by imprinting a phase profile across the incoming wavefront, resulting in a far field diffraction pattern yielding the desired illumination pattern. These multiple independent beamlets can be directed to different regions, and depths, on the sample 160, simultaneously. When the galvanometers are scanned, each individual beamlet can sweep across its targeted area on the sample 160, generating a fluorescence that can be collected by a single pixel detector (e.g., photomultiplier tube). As multiple regions of the sample can be illuminated simultaneously, the resultant "image" can be a superposition of all of the individual images that would have been produced by scanning each separate beamlet individually {See, e.g., Figures IB and 1C).
[0071] To demonstrate the exemplary multiplane imaging system, a structural imaging of a brine shrimp, Artemia naupili, was performed, collecting its intrinsic autofluorescence. A traditional serial "z-stack", with seven planes, was acquired by moving the objective 50 μπι axially between each plane {See, e.g., Figures ID). Next, a serial z-stack was acquired with seven planes with 50 μπι separation, but with the objective fixed and the axial displacements generated by imparting a lens phase function on the SLM. {See, e.g., Figures IE). Figures IF and 1G, show the arithmetic sums of all of the images in Figures ID and IE, respectively. The exemplary SLM can be used to generate all seven axially displaced beamlets simultaneously, and can scan them across the sample. {See, e.g., Figure 1H). In some exemplary embodiments of the present disclosure, a Bessel or an Airy beam can be used to directly provide an extended depth of field. {See, e.g., References 19 and 59). While these beams can give a similar result about for the shrimp, or similarly transparent samples, they can provide near uniform intensity across the entire depth of field, while the exemplary approach can facilitate independent power adjustment of each depth. This is shown in Figure II where the power directed to a single plane (e.g., 50 μπι) is selectively increased, and the signal, only for features at that depth, is increased. This flexibility can be beneficial for inhomogeneously stained samples, and for multi- depth in-vivo imaging in scattering tissue, as is illustrated below, and may only be possible with independently configurable beamlets. Additionally, by illuminating only a select number of planes, the "density" of the resulting images can be controlled; for example, how many sections used to contribute to the final image can be controlled.
[0072] As shown in Figures 2A-2H, the functional imaging application of the exemplary system 205 is shown. In-vivo two-photon imaging of layer 2/3 (e.g., L2/3) in the primary visual cortex (e.g., VI), at a depth of 280 μηι, at 10 Hz, was performed in an awake head-fixed mouse that expresses the genetically encoded calcium indicator, GCaMP6f {See, e.g., Reference 14 ). The exemplary SLM can be configured similar to that illustrated in Figure 1C, to expand the effective sampled area. To increase the imaged area, the beam can be laterally split, creating two beams of equal power with an on-sample separation of -300 μπι centered on the original field of view ("FOV"). {See, e.g., Figure 2A). Figures 2B-2D, , show images of the standard deviation ("std. dev.") of intensity across the acquired time-series image sequences that result from scanning each of these displaced beams individually, and their arithmetic sum, respectively, while Figure 2E shows the std. dev. image acquired when both beams can be simultaneously scanned across the sample. The lower subpanels show detected source ROIs from the images, with the element 210 and 215 reflecting the originating source FOV. The areas contained in the rectangles 220 and 225 in Figures 2B and 2C, respectively, highlight the area that can be contained in the FOV of both beamlets, and thus their ROIs can be present twice in the dual plane image.
[0073] Figures 2F and 2G show representative extracted fluorescence time series data from the detected ROIs (e.g., 40 out of 235 shown). The same ROIs are displayed in both Figures 2F and 2G, with the same ordering, to facilitate direct comparison of the single and dual plane traces. In Figure 2F, the FOVs were collected sequentially - that can be one at a time, and thus the traces 230 were collected at a different absolute time than the traces 235. Traces 240 and 245 traces shown in Figure 2F, however, were collected simultaneously. Because of the relative spatial sparsity of active neurons {see, e.g., Figure 2E, bottom), many of the ROIs can be separable even in the overlaid dual region image, and fluorescence time series data can be easily extracted using exemplary procedures. However, some ROIs show clear overlap, and more sophisticated procedures, such as independent component analysis ("ICA") {see, e.g., Reference 39), or a structured matrix factorization method ("C MF"), can perform better at extracting the activity. The traces shown were extracted using an exemplary CNMF method, which will be highlighted and discussed in greater detail below. {See, e.g., Figure 4). {See, e.g., Reference 46). In examining the traces in more detail, the overall effective SNR can be high, in both the sequentially acquired data and in the simultaneously collected dual region data, which can facilitate events to be easily detected by exemplary automated procedures. Because the collected multi-region image can be the arithmetic sum of the two single region images, the detected ROIs from the single region image can be used as strong prior knowledge for source localization. This can be leveraged to produce very good initial estimates on the likely number of independent sources, and their spatial location. This prior knowledge can be extremely useful for unmixing complex overlapping signals, and can increase the overall performance of the source extraction in the mixed images. The "uniqueness" of signal recovery can be examined by looking at the ROIs that appear twice in the dual plane image, and can display identical dynamics. Two exemplars are highlighted (e.g., element 250) shown in Figure 2G, and expanded in Figure 2H; traces 255 and 260 show the source copy generated from each beamlet, and the extremely high correlation between the extracted traces (e.g., R>0.985). These duplicative sources can be easily removed from the total independent ROI count; first, because the positioning of the FOVs can be deterministically controlled, and a priori known, which regions can be the shared, and where the components will appear in the image can be determined. It can be noted that even without that such knowledge, such source ROIs can be identified by their extremely high cross-correlation.
[0074] Compared to the original FOV, the dual region image can include signals from a significantly larger total area. The single region FOV was approximate exemplary 380 μπι x 380 μπι, and captured 1.45xl05 μπι2, while the dual region image captured signals from
2.66x105 μπι2 (e.g., twice the FOV, minus the overlapped region), representing an about 84% increase in interrogated area, with no loss in temporal resolution. With the exemplary system, the exemplary maximal useful lateral displacement of each beamlet from the center of the FOV approximately 150 um. {See, e.g., Figure 7C and 7D). Because the SLM can be flexible in its ability to address arbitrary subregions of the FOV, alternate SLM approaches can be performed. For example, the effective frame rate of a system can be doubled by creating two laterally displaced beams, with an angular spread one half of the FOV, in the direction orthogonal to the fast axis of the galvos. By scanning the galvos over the middle 50% of the image, the displaced beams can still illuminate the entire FOV, though the total number of lines scanned can be halved, which can double the overall frame rate. Alternately, two or more small subregions enclosed within a single larger FOV can be scanned simultaneously. Under all of these paradigms, it can be noted that the splitting can facilitate increases in effective frame rate while keeping the dwell time per pixel of each region higher than what would be possible if the regions were sequentially scanned, which can increase the overall signal collected from each region. [0075] While the lateral imaging procedures can increase imaging performance, the full power of multiplexed SLM imaging lies in its ability to flexibly address axially displaced planes, with independent control of beamlet power and position. An exemplary defocus aberration can be introduced to the wavefront, which can shift the beam focus away from the nominal focal plane. Higher order axially dependent phase terms can be included to offset the effects of higher-order aberrations, and facilitate "prism" shifts as well, which can add flexibly by facilitating for lateral displacements. The exemplary system can provide high performance, and gives approximately 500 μιη of axial displacement while maintaining total collected two-photon fluorescence at greater than 50% of that generated at the objective's natural focal plane. {See, e.g., Figure 7B). This can provide a considerable range for scanning, and it does so with high speed inertia-free focusing (e.g., less than 3 ms, as shown in Figure 11). Because the objective may not be moving, there may be no acoustic noise, and there can be no vibrations transferred to the objective or sample, both of which can perturb animal preparations, or for strong vibrations, even damage the objective. Using the exemplary SLM, multiple axial planes can be addressed simultaneously - multiple lens phase functions can be combined, and imprinted onto the incoming beam by the SLM. Figures 3 A and 3B show two examples of conventional "single- plane" two-photon images in mouse VI, the first approximately 170 μπι below the pial surface, in L2/3, and the second 500 μπι below the surface, in L5. These images were acquired with SLM focusing; the objective's focal plane was fixed at a depth of 380 μπι, and the axial displacements were generated by imposing the appropriate lens phase on the SLM. At each depth, functional signals at 10Hz were recorded. Figure 3E shows some representative extracted fluorescence traces from L2/3 {e.g., element 305), and L5 {e.g., element 310) (e.g., 154 and 196 total ROIs detected across the upper and lower planes, respectively).
[0076] The exemplary SLM was then used to simultaneously split the incoming beam into two axially displaced beams, directed to cortical depths of 170 μπι and 500 μπι, and scanned over the sample at 10 Hz. {See, e.g., Figure 3D). Because the mouse cortex can be highly scattering {see, e.g., References 17 and 33), significantly more power can be needed to image L5 than L2/3, and the power directed to each plane can be adjusted such that the collected fluorescence from each plane was approximately equal. The effective collected two photon signal can also depend on the overall efficiency of the SLM in redirecting the light to axial positions other than the designed focal plane of the microscope. Multiple factors can contribute to this, from optical parameters of the two-photon microscope, such as the bandwidth of the laser source and the effective numerical aperture ("NA") of excitation, along with the objective magnification, to SLM specific device parameters such as the number distinct phase levels, the pixels density, and the fill factor of the device. In designing the exemplary system, it can be beneficial to holographically deflect light over a span of approsimately500 μπι axially, while also maintaining clear subcellular resolution and high two- photon efficiency.
[0077] The exemplary measured efficiency curve can be slightly asymmetric (see, e.g., Figure 7B), and shows that the beam can be projected from 200 μπι beyond (e.g., deeper) the focal plane of the objective, to 300 μπι above (e.g., shallower) than the objective's focal plane while maintaining strong two-photon excitation. This curve can be relied upon as the objective's focal plane depth to a specific position, and can be set to optimally deliver power to the chosen targeted planes for simultaneous imaging, or to compute the power used for a good signal at various multiplane combinations. For the particular pair of planes acquired, as shown in Figures 3 A and 3B, scattering alone can dictate that the L5 plane can utilize approximately six times the input power than that of the L2/3 image to match the signals. With the microscope' s natural focal plane set to 380 μπι, the SLM efficiency for redirection can be the same for both planes, and the dual beam hologram can be computed with an approximately six-fold increase in intensity for the lower layer.
[0078] Scanning these beamlets over the sample, the dual plane image can be collected, which is shown, along with the ROIs, in Figure 3D, which can correspond very well to the arithmetic sum of the individual plane images. In Figure 3F, thirty representative traces (e.g., out of 350 source ROIs) are shown, which illustrate spontaneous activity across L2/3 (e.g., element 315) and L5 (e.g., element 320), with a high SNR. The ordering of traces is identical with that of Figure 3E, which facilitates comparison of the signals detected with the conventionally acquired single plane images. The lightly shaded regions 325 in Figures 3E and 3F are enlarged and shown in Figures 3G and 3H. The traces 330 and 335 can be the signals extracted from the ROI using the CNMF method, which both optimally weights individual pixels and denoises the signal, while the underlying trace 340 shows the raw signal from the ROI. The raw signals show very clear events, with high SNR, and the CNMF traces can be even cleaner. Further exemplary zooms of small events are shown in Figure 31, which shows expanded views of the small peaks labeled i-iv on Figures 3G and 3H. [0079] With a cursory examination of the ROIs in the dual plane image, it can be clear that there can be significant overlap between a number of sources, as expected by collecting fluorescence from both areas with a single pixel detector ("PMT"), without specific efforts to avoid such conditions. There have been many hardware strategies implemented to avoid such "cross-talk", from temporal multiplexing (see, e.g., Reference 15), to multiple array detectors (see, e.g., Reference 32), but these can be relatively complex, and while they can reduce, they never completely eliminate signal mixing. A software based approach can be utilized (See, e.g., Reference 46).
[0080] A generalized biophysical model can be used to relate the detected fluorescence from a source (e.g., neuron) to the underlying activity (e.g., spiking) (See, e.g., References 61 and 62). This can be extended to where the detected signal (e.g., fluorescence plus noise) in each single pixel can come from multiple underlying sources, which can produce a spatiotemporal mixing of signals in that pixel. The exemplary goal then can be, given a set of pixels of time varying intensity, infer the low-rank matrix of underlying independent signal sources that generated the measured signals. The non- negativity of fluorescence and of the underlying neuronal activity can be taken advantage of, and the computationally efficient constrained non-negative matrix factorization methods can be used to perform the source separation; thus the label of C MF.
[0081] To extract signals from the multiplane image, the exemplary procedure can be initialized with the expected number of sources (e.g., the rank), along with the nominal expected spatial location of the sources as prior knowledge, as identified by running the procedure on the previously acquired single plane image sequences. For the single plane images, the complexity, and number of overlapping sources can be significantly less than the multiplane images, and the procedure works very well for identifying sources without additional guidance. The
effectiveness of this exemplary procedure, as applied to multiplane imaging, is shown in Figures 4A-4F. This can also be compared against the exemplary "best" human effort at selecting only the non, or minimally overlapping, pixels from each source, and against independent component analysis ("ICA"), which has previously proved very successful in extracting individual sources from mixed signals in calcium imaging movies(See, e.g., Reference 39). Figures 4A-4C show progressively more complex spatial patches of the dual plane image series with Figure 4D showing the location of each patch in the overall image. The overall structure shown in Figures 4A- 4C is identical. In each figure therein, the uppermost row of images show the maximum intensity projection of the time series, with the subsequent images showing, representative time points where the component sources can be independently active. The leftmost column of images shows the weighted mask, labeled with the ROI selection scheme (e.g., binary mask from maximum intensity projection, human selected non-overlapped, ICA, CNMF) that can produce the activity traces presented immediately to the right of these boxes. Therefore, the pixel weighting is shown adjacent to Figure 4D. The extracted CNMF traces can include both the full CNMF extracted trace 405, and for comparison, trace 410, the signal extracted considering only the CNMF produced spatially weighted ROI, without taking into account the temporal mixing of other sources into those pixels. In simple cases (see, e.g., Figures 4A and 4B), the human selected non-overlapped regions appear to select only a subset of the events seen in the combined binary mask. In more complex cases (see, e.g., Figure 4C and Figures 10B-G), where multiple distinct sources can overlap with the chosen source, the non-overlapped portion may only contain a few pixels, yielding a poor SNR, or those pixels may not be truly free from
contamination, yielding mixed signals. In regions where the non-overlapped portion can be identifiable, this can be used as a Reference to evaluate the other two procedures.
[0082] The ICA extracted sources can then be examined. ICA can identify the sources automatically, without human intervention, and does so quickly. For cases where the number of sources in space can be low, and there can be "clean" non-overlapping pixels with high SNR (see, e.g., Figures 4A and 4B), the extracted components can be spatially consistent with the known source location (e.g., top row of images). In many cases, they can include a region of low magnitude negative weights, which can appear to spatially overlap adjacent detected sources, presumably because this can decrease the apparent mixing of signals between the components. This can lead to unphysical minor negative transients, but these can be easily ignored with simple thresholding. For the complex overlapping signals in the exemplary experiment, ICA can routinely fail to identify human and CNMF identified source components, and appears to have less clean separation of mixed signals (e.g., activity traces shown in Figure 4A, and the complete failure of ICA to identify ROI2 shown in Figure 4C). Additionally, in order to maximize the performance of ICA, the acquired image can be tiled into smaller sub-images, with fewer component sources, for ICA to give reasonable performance. This can be simple to implement, but even in these instances, the total number of detected independent components with activity traces that look like real signals (e.g., appear "cell-shaped") and have temporal structure readily identifiable as physical (e.g., transients with fast leading edges, and characteristic longer decays) can be lower than what either a human, or CNMF methods identify {See, e.g., Figures 9A and 9B). {See, e.g., Reference 46). Nevertheless, ICA can find a significant number of sources, and does so automatically, an advantage over manually choosing ROIs. The graph shown in Figure 9A illustrates the exemplary correlation coefficient between the AF/F extracted from CNMF and NOL for a total of 250 ROIs, for the data shown in Figures 3A-3I and 4A-4F. The dashed 905 line indicates the median of the correlation coefficients. The ROI contour map is also plotted.
[0083] Figures 9B-9G show exemplary ROIs with various correlation coefficient between CNMF and NOL. For each case, the ROI is labelled as 1, and its adjacent ROIs (e.g., potential contamination sources) are shown in Figure 9 A. To better evaluate the correlation coefficient of the signal extracted from the CNMF and NOL, both signals are plotted. Using the ROI contour in the CNMF but with uniformed pixel weighting and without unmixing treatment, the extracted AF/F trace 910 is plotted, superimposed onto the traces extracted from CNMF. The signals extracted from their adjacent ROIs (e.g., using CNMF) are plotted in green.
[0084] On inspection of the CNMF traces, many things can be seen. First, for the extracted traces that are automatically denoised, the exemplary model can facilitate this in a
straightforward fashion. Second, the identified sources can be well separated. Comparing the CNMF trace to the non-overlapped trace, a very high correspondence can be seen, especially when the respective non-overlapped source SNR can be high. As shown in Figure 4A, it is possible to see that the CNMF can be better than ICA at eliminating cross-talk between the two overlapping ROIs. This trend can be seen across all of the examples shown, and can generally be conserved over all ROIs. The CNMF traces can be compared against the human selected non- overlapping traces 415 shown in Figure 4E. This exemplary graph shows the cross correlation between 250 CNMF sources, and the non-overlapped portion of that source (e.g., NOL) related to it. The correlation can be high, as would be expected if the CNMF traces accurately detect the underlying source. The majority of the signals show a correlation coefficient of less than 0.95, and the mean correlation coefficient can be 0.91, including all outliers. It can be noted that the distribution of coefficients can be strongly asymmetric, and it can contain some notable outliers.
[0085] Examining the underlying traces of these outliers, the source of the poor correlation between the NOL trace and the CNMF trace can be identified. {See, e.g., Figures 10B-10G). In these exemplary cases, it can be seen that the NOL ROI can consist only of a few pixels, and can be extremely noisy. The related CNMF ROI can be significantly larger and has much less noise. Examining the neighboring, or overlapping, sources that surround the NOL (e.g., and CNMF) component, sources can be seen with very different activity than that extracted in the CNMF trace, but clearly are present in the NOL signal. This phenomenon can be present in all of the lower correlation components. Because of this, the graph can be interpreted in another way. The fraction of cells with low correlation between the NOL and CNMF components can be exactly the sources that simple signal extraction procedures cannot cleanly extract the true underlying activity, they can be the cells that benefit from CNMF. Without CNMF, extracted signals can falsely show high correlation between sources that can be imaged into the same region in the dual plane image. In the dual plane image shown in Figure 3, it can be found that approximately 30% of the cells have correlations below 0.9. The exact fraction of cells with significant contamination can depend on the sources sparsity and targeted areas, but there may always be many cells that can show some overlap, and would benefit from a high performance unmixing strategy, like CNMF.
[0086] The power of the exemplary simultaneous multiplane imaging and source separation approach can be seen in Figures 5A-6E, where interlaminar correlations can be recorded, which can be used for understanding microcircuit information flow in the brain, as well as can evoke functional responses across cortical layers. As shown in Figure 5 A, 150 fluorescence activity traces 505 (e.g., out of approximately 350) showing spontaneous activity in L2/3 and L5 neurons in VI, collected at 10 Hz (e.g., FOV and cells as in Figures 3A-3H) are shown. In 13 minutes of imaging, over 20,000 events were detected across the population of active cells (e.g., 250 ROIs). The significant correlations between cells in L2/3 and cells in L5 can be computed, and the interlaminar correlation map in Figure 5B can be displayed. One of the high correlation pairs (e.g., projected lateral displacement of approximately 67 μπι) is shown as element 510 (e.g., dark outlines) in Figure 5C. A scatterplot showing the correlation coefficients between pairs with respect to projected lateral displacement is shown in Figure 5D. It can be noted that for the spontaneous activity in this recording, little dependence in the average correlation between cells in L2/3 and L5 on lateral distance between cells (e.g., inset, Figure 5D) can be seen.
[0087] The data shown in Figures 3 A-3I and 5A-5D were taken with two planes with inter- plane spacing of 330 μπι, which can be far from the limit of the exemplary procedure. Both the number of planes, and the total extremal range between planes over 500 μπι, can be extended with a high S R. (See, e.g., Figures 10A-10O). In these additional experiments, the total performance and number of ROIs was limited by poor expression of the indicator in the mice (e.g., limited number of cells expressing strongly, combined with high non- specific background fluorescence). A shown in Figures 10A-10H, simultaneous three-plane imaging, again in mouse VI, is shown, and some representative activity traces from each of the planes is highlighted, as well as clean source separation using CNMF for a region of interest with partial spatial overlap from each of the planes can be shown. Deeper, extended, axial planes can be further
demonstrated by imaging three planes at cortical depths of 130 mm, 430 mm and 640 mm, at 10Hz, again with a high SNR. (See, e.g., Figures 10I-10O).
[0088] In a further set of experiments, visually evoked activity across L2/3 and L5 was examined. Drifting gratings were projected to probe the orientation and directional sensitivity (e.g., OS and DS,) of the neuronal responses. This paradigm was chosen because drifting grating produce robust responses in VI (see, e.g., Reference 40), can be used frequently in the community (see, e.g., References 27 and 52), and can be used to examine the performance of the exemplary imaging procedure in a functional context. Figure 6A shows some representative extracted traces 605 and 610 from cells with different OS, along with an indicator showing the timing of presentation of the preferred stimulus (e.g., traces 605 and 610 were acquired in the dual-plane paradigm). In this exemplary experiment, the activity of each plane individually, for multiple trials, was recorded, followed by a brief interval with no visual stimulation, followed by recording the dual plane images.
[0089] During the entire recording period, this mouse had particularly strong spontaneous activity in nearly all of the cells in the FOV. Many cells showed strong and consistent orientation tuning across single-plane trials (e.g., 75 out of 260 cells). For these cells, the computed single-plane DS and OS were compared to those computed from the same neurons, but from the dual-plane image series. If the dual plane images had increased noise, or if overlapping sources could not be cleanly separated, a decrease can be expected, or an OS can be altered. This was neither the case on the population, nor single cell level, as shown in Figures 6 B and 6C. Figure 6D shows the exemplary standard dev. dual-plane image, and extracted ROI contours. The two small boxes 625 on the contour image indicate two pairs of cells, with significant spatial overlap in the dual-plane image. This is illustrated in more detail in Figure 6E, where the DS of these cells is shown to ensure that the source separation procedure cleanly extracted the functional activity. It can be clear that the functional activity of even strongly overlapping cells can be fully separated, without cross contamination.
Exemplary Discussion
[0090] Successful simultaneous 3D multilayer is in-vivo imaging is shown with a hybrid SLM multibeam-scanning approach that can leverage spatiotemporal sparseness of activity and prior structural information to efficiently extract single cell neuronal activity. The effective area that can be sampled can be extended, multiple axial planes can be targeted over an extended range, greater than 500 μπι, or both, at depth within the cortex. This can enable the detailed
examination of intra- and inter-laminar functional activity. The exemplary procedure can be easily implemented on any microscope, with the addition of a SLM module to the excitation path, and without any additional hardware modifications in the detection path. The regional targeting can be performed remotely, through holography, without any motion of the objective, which can make the exemplary procedure a strong complement to 3D two-photon activation {See, e.g., References 44, 45, and 51).
Exemplary Comparisons to Alternative Methods
[0091] Many imaging modalities exist today. Some of the simplest systems that provide volumetric imaging can combine a piezo mounted objective with resonant galvos. Optically, these systems have high performance throughout the focusing range, as all components can be used in their best-designed positions. A critical component for determining the imaging rate can be the speed of the piezo and how fast can the objective be translated axially. This can be related to the resonant frequency of the combined piezo-objective system, and the maximum forces and accelerations facilitated in the system. For deep imaging in scattering tissue, the fluorescence collection efficiency can scale as—^, with M being the objective magnification. (See, e.g.,
Reference 6). Unfortunately, the combination of high NA, and low magnification, mean that the objectives can be large, and heavy. This large effective mass lowers the resonant frequency of the combined piezo/objective system, and necessitates significant forces to axially move quickly, as well as lengthens the settle times (e.g., approximately 15 ms). As such, while 2D imaging rates can be high, volumetric imaging can be slower, with volumetric rates less than lOHz (e.g., 3 planes, sequentially).
[0092] The "throw" - the distance with which piezo can travel, can also be limited, with the current state-of-the-art systems offering 400 μιη of total travel, although most systems have significantly less. The settle time can also lead to lower duty cycles as imaging cannot take place during objective settling. Compared to piezo-based systems, the exemplary SLM-based system/apparatus has significantly greater axial range, and couples no vibrations into the sample. It can be possible to currently scan at least three planes simultaneously at 10 Hz using traditional galvanometers, with over 500 μπι total separation between the outer planes (see, e.g., Figures 10A-10O), and the speed could be immediately tripled by installing a resonant galvanometer on the exemplary system. Additionally, change to a different axial plane can be performed very rapidly, in approzimately3ms, which can minimize lost time, and can facilitate axial switching within a frame, with little data loss (See, e.g., Figures 11 A-l 1C).
[0093] Remote focusing has also been used for faster volumetric imaging, either with the use of a secondary objective and movable mirror (see, e.g., References 9 and 10), or with
electrotunable lenses (See, e.g., Reference 23). While both have higher performance than piezo mounted objectives, neither has yet demonstrated the ability to facilitate in-vivo functional imaging at the axial span illustrated herein. Remote focusing with movable mirrors can scan with minimal aberration of the PSF, but for cell targeted imaging, defocusing induced aberrations may not be significant, neither in the exemplary procedure, nor for electrotunable lenses. For applications where a perfect PSF can be paramount, remote focusing with a mirror can offer better optical performance, but benefits from careful alignment and engineering, and may not be beneficial for somatic calcium imaging. The electrotunable lens represents perhaps the most cost effective solution for high performance fast focusing, and can be inserted directly behind the objective. But in this position, it affects any beam that passes through it, so it can complicate combining two-photon activation with imaging. A better solution can be to place it in a conjugate plane to the back focal plane of the objective - exactly the same nominal position as the exemplary SLM - and then it can control the imaging beam alone. While an SLM can be more costly than the electrotunable lens, it can still be a small expense relatively to the cost of any two-photon microscope, and offers faster settle times (see, e.g., Figures 1 1 A-l 1C), as well as multiplexed excitation. [0094] While fast sequential imaging strategies such as acousto-optic deflector ("AOD") systems offer good performance, with the current state-of-the-art 3D AOD systems currently providing high performance imaging over relatively large volumes of tissue {See, e.g.,
References 31, 43, and 50). Unfortunately, these systems can be very complex and expensive, with the cost of these systems at least a few times that of conventional two-photon microscopes, which severely limits their use. They also can be very sensitive to wavelength, benefits from extensive realignment with changes in wavelength. The scanning range of most AOD systems can be less than most systems, {see, e.g., References 29, 34, 50 and 53), with only the most strongly chromatically corrected variant exceeding the exemplary demonstrated range. {See, e.g., Reference 31). With the addition of the same chromatic correction optics to the exemplary system, the addressable volumes can be similar. An additional complication of any point targeting strategy, like AOD systems, can be that sample motions can be significantly more difficult to treat. With raster scanning, shifts in the XY plane can be easily detectible, and treatable with well-established correction procedures. Point targeting systems, on the other hand, need to densely target at a few ROIs in the sample to create a fiducial that can be used determine the magnitude and direction of motion. These fiducials need to be consistently visible, for closed loop correction.
[0095] A bigger limitation of all of these serially scanned systems can be that all can be nearing the fundamental limits of their speed, and further increases in performance can be likely only possible with multibeam systems. For example, in AOD based systems, while the "speed- limit" can be given by the speed of sound in the material, a more practical limit can be set by the dwell time used per target to acquire signals with sufficient S R. Single beam large FOV galvanometric scanning systems suffer from this as well, if one maintains high frame rates, the linear speed of travel of the excitation spot across the FOV can be high, resulting in low dwell times per unit distance on the sample. To maintain SNR at useful levels, finite dwell times can be beneficial. As fluorophore saturation ultimately can limit the maximum emission rate regardless of excitation intensity (e.g., increases in intensity simply cause photodamage, bleaching, and reduced spatial resolution). The current "best" functional indicators, like
GCaMP6, have high two-photon cross sections (see, e.g., Reference 14), and excitation saturation can be easily reached with modern laser systems. This can affect all single beam systems that use fluorescence as the optical readout. [0096] An exemplary procedure to increase performance, and take advantage of available power, while maintaining sufficient dwell times for high sensitivity, can be to have multiple beams targeting the sample. Spatially multiplexed strategies have been used before (see, e.g., References 6, 8, 21, 32, and 37), but with very limited success for imaging neuronal activity in scattering samples. Imaging in scattering samples can produce fluorescence, which can scatter extensively, limiting the ability to "assign" each fluorescence photon to its source (See, e.g., References 4, 3 and 32). Under these conditions, a more successful approach has been to time multiplex each separate excitation beam (See, e.g., References 4, 15, 19 and 20).
[0097] The standard lasers used in two-photon imaging operate at approximately 80 MHz, with the individual pulses separated by 12 ns, and many standard fluorophores used in biological imaging have fluorescence lifetimes of less than or equal to 3ns. Thus, each single laser pulse can produce a short burst of fluorescence that can extend strongly for only 25% of the "dark time", rendering the system's duty cycle low. With an appropriate combination of beam splitters and delay lines, the repetition rate can be effectively increased by a factor of 4, to approximately 320 MHz. with approximately 3 ns between pulses. As these four beams traverse independent paths before entering the microscope, each beam can be adjusted to a different area and depth, and can be used for multilayer (see, e.g., Reference 15) or multi-area imaging. (See, e.g., Reference 58). For multilayer imaging, the exemplary system can exceed the demonstrated FOV and axial range of published implementation of temporally multiplexed beams, without customized electronics for time-stamped photon counting, and with significantly greater flexibility and speed in choosing targeted depths. Other multibeam or temporally multiplexed strategies to be can be complementary methods that can be leveraged to further increase the exemplary system performance. Exemplary Phase Only SLMs For Beam Steering
[0098] The performance of the exemplary system can depend on the SLM, a 512x512 pixel phase only device that performs beam-splitting by imparting phase modulation on the incoming laser pulse, and uses diffraction to redirect the beams to their targeted site. The efficiency of any diffractive process can be affected by many things (e.g., the fill factor and the effective number of phase levels per "feature"). The controllable pixel fill factor of the exemplary SLM can be 82%, and thus, even with "perfect" holograms, the maximum power throughput can be reduced compared to a simple mirror. Taken together, these factors can lead to an effective power throughput on the SLM module ranging from approximately82% to approximately 40%, reducing the total light available for imaging. In exemplary experiments, the efficiency has not yet been a limitation for multiplane imaging, but it can be for imaging multiple planes in L5 and beyond. Compared to a conventional system with a piezo mounted objective, this can seem like a significant loss, however, it can be far less of a penalty than for multi-AOD systems, which give less than 20% total power throughput.
[0099] Another important consideration for any diffractive device can be that they can be inherently chromatic (e.g., the deflection can depend on the wavelength of the light, laser sources for multi-photon imaging may not be strictly monochromatic (e.g., the ultrafast laser pulse has a characteristic bandwidth, with a minimal range dependent on the shape of the pulse. In the exemplary system (e.g., Coherent Mira-HP) this can correspond to a FWHM of 6.5 nm at 940 nm for the approximately 140 fs pulses from the laser. With a 25X objective, and an SLM induced lateral displacement of 150 μπι from the origin for the central beam, it can be found that the spectral FWHM colors to be deflected ±1 μπι from the target position.
[00100] If the effective excitation NA can be large, and the PSF correspondingly small, these displacements can be comparable in size with the FWHM of the monochromatic PSF, which both blurs the image, and reduces the peak focused intensity at such displacements. With shorter pulses, performance fall off can be more severe, and without additional correction elements, pulse widths shorter than lOOfs may not be recommended. This chromatic dispersion can be reduced with the incorporation of a custom shaped high dispersion optical element, (see, e.g., Reference 31), to improve the performance over a wider lateral range, and for shorter pulses. Nonetheless, even without efforts to reduce dispersion, the FOV of a meaningful amount can be extended, even without correction, and within that range, rapidly, and flexibly.
[00101] For axial displacements on the exemplary system, the effect of chromatic dispersion can be markedly less. For a diffractive lens, the dependence of focal length with wavelength can be /(Λ) « fo (j-), which can correspond to an axial shift between the FWHM points of approximately 1.1 μπι for a -200 μπι displacement, and approximately 1.6 μπι for a +300 μπι displacement. The objective back aperture can be filled to have an NA of 0.45, and with these settings, the FWHM of the nominal monochromatic PSF can be approximately 6.5 μηι, so the chromatic axial displacement may only be a small fraction of the PSF, for the entire span of axial separations demonstrated here. If finer axial resolution can be beneficial, which can benefit from larger excitation NA, the axial range with high performance can be correspondingly smaller. For a given SLM setting, the relative displacement it produces, and the resultant axial PSF can, both scale as NA2. For example, at an excitation NA of 0.9, only 120 μπι of axial separation can be possible with the same relative performance. For the exemplary task, fast multiplane calcium imaging with single cell resolution, the performance can be excellent over 500 μπι.
[00102] The efficiency of this procedure can be improved by better SLMs (e.g., devices that have higher fill factors, increased pixel number and increased phase modulation. Increased pixel counts can increase the number of levels available across any feature, and can increase the diffraction efficiency. For nearly all applications, once the pixilation facilitates greater than or equal to 8 levels across a ramped 0 to 2π transition, the efficiency can be very high. For arbitrary patterns of excitation, it may not be possible to predict how many overall pixels can be needed on the device to achieve this, but for the simple lateral or axial deflections, this can correspond to SLMs with approximately 1500 pixels across, which can keep the overall module power efficiency above 80% for the current fill factor, and higher with improved devices. More pixels can also increase the possible range of deflections; however, this may not extend the range without better compensation of chromatic dispersion, which can be one reason why existing, higher pixel devices, may not be used. The total size of the device may not be critical, and the pixel density may not need to be increased; just the total number of pixels. Large pixels have an added benefit of increased power handling, and generally higher fill factors, because the electronics and insulating barriers between neighboring pixels can be somewhat fixed in size, whereas active areas may not be. Larger pixels can result in larger phase features, which can reduce the maximum angles of deflection, but this can be easily compensated with altering the magnification of the post-SLM telescope system, which can be beneficial to properly map the SLM to the back focal plane of the objective.
[00103] Another area where efficiency can be increased can be by combining adaptive optics methods with the exemplary targeting. SLMs can be a natural element for corrections of both system, and sample aberrations {See, e.g., Reference 30).
Exemplary Volumetric Imaging [00104] The overall speed of multiplane imaging can be increased with additional exemplary procedures. Because of the systems sensitivity, activity with high S R can be detected. As such, the nominal dwell time per pixel can be reduced, and enough photons for effective detection can still be collected. By transitioning the exemplary microscope from conventional galvanometers to resonant galvanometers, the imaging can be sped up by at least a factor of three. Combined with the fast switching time of the exemplary device, large volumes of neural tissue can be imaged at high speed. The exemplary strategy can be to rapidly interleave multiplane images in successive scans to generate a complete picture of neural activity.
[00105] The exemplary SLM was chosen specifically because it can be controlled via low latency PCIe bus transfers, increasing speed, and giving deterministic latency for pattern changes. This device was optimized for fast hologram transitions by increasing the backplane drive voltage, utilizing more than 2π phase stroke, and the computation of optimized
intermediate holograms. Taken together, they dramatically increase the effective switching speed of the device, and facilitate beam retargeting in less than 3ms. Figure 1 IB shows a graph that illustrates the performance. Two sets of planes in 3 ms can be fully transitioned between, with usable fluorescence produced in the new set of planes in 2 ms. Imaging at 30 Hz, with three sets of 3-plane images, can be performed. The fast transition times can ensure interframe switching while maximizing duty cycle. This can facilitate nine axial sections, with an overall volume rate of 10 Hz, over large regions of the cortex, and significant depth. {See, e.g., Figure 11C). The imaged volume can also be increased by scaling up the number of independent beams entering the SLM modules, and incorporating various spatial and temporal multiplexing strategies {See, e.g., References 15 and 58). In these cases, the addressable volume or speed increase can scale linearly with the number of beams entering the SLM module(s).
[00106] Further volumetric imaging in scattering tissue can be promising, and SLM- based multiregion imaging can be but one implementation of a general strategy of computationally enhanced projective imaging, which can make possible the ability to interrogate neurons over a very large area, with high temporal resolution and SNR. Projective imaging can be extremely powerful, and has led to tremendous advances in medical imaging, such as computerized tomography ("CT"), positron emission tomography ("PET"), and magnetic resonance imaging ("MRI"). Many of these methods suffer from issues familiar to neuroscientists aiming at imaging activity in the brain. For x-ray CT scanning and PET, "photodamage" can be a serious concern, while for MRI, speed can be limited. For those modalities, advanced computational and statistical procedures, ideally suited to the particular imaging "task", shape the overall strategy.
[00107] Basic knowledge of the underlying physical structure and sparsity can be used to define constraints for the recovery of the underlying signal, and can facilitate higher fidelity reconstruction and increased imaging speed. {See, .e.g., References 7, 11, 14 and 57). These same principles apply here. The spatial distribution of neurons may not be dense and uniform everywhere, and neuronal signals may not be consistently highly synchronized; this can produce spatiotemporal sparseness that can be leveraged for efficient source separation of mixed signals. With the exemplary multiplexed SLM system/apparatus the number of areas simultaneously illuminated can be deterministically controlled such that there can be direct control over the effective number of sources, in contrast to alternate extended two-photon approaches, like Bessel beam scanning {see, e.g., References 11 and 59), where the sample alone controls the complexity of the signal.
[00108] Just as contrast agents dramatically have improved performance in other imaging modalities, this exemplary procedure can include advances in functional indicators. GCaMP6 has very low basal fluorescence, which may increase only with neuronal firing (e.g., the dark of space, punctuated by stars of activity). This has advantages for the signal unmixing and SNR, as background fluorescence can be minimal. As development continues on these and other probes, more sites can be targeted simultaneously.
[00109] There can be many issues that can be addressed by high speed volumetric imaging. First, for example, the simple increase in neurons monitored in the local circuit can greatly increase the chances for capturing the richness of variability and dynamics in cortical processing {See, e.g., References 2, 3 and 28). For example, what can be the organization of functionally or behaviorally relevant ensembles in cortical columns? How do upstream interneurons affect downstream activity and synchrony, and output {See, e.g., References 23, 25, 39)? Without the ability to probe interlaminar activity simultaneously, answering these questions definitively can be very difficult, if not impossible. While somatic imaging has been described herein, the exemplary procedure functions well for imaging dendrites, or dendrites and soma {see, e.g., Reference 46) for dendritic source separation with CNMF. The extended axial range of the exemplary procedure can facilitate exploring L5 soma and their apical tufts simultaneously, and can give direct insight into the role of dendritic spikes and computation in neuronal output. {See, e.g., References 35 and 55). The SLM-based multiplane imaging can be a powerful procedure for addressing these and other questions that can benefit from high speed volumetric imaging with clear cellular resolution. The exemplary system can be flexible, easily configurable and compatible with most existing two photon microscopes, and can provide new insights into cortical computation and function across multiple layers.
Exemplary Experimental Procedures
Exemplary Animals and Surgery
[00110] All experimental procedures were carried out in accordance with animal protocols approved by Columbia University Institutional Animal Care and Use Committee. Experiments were performed with C57BL/6 wild-type mice at the age of postnatal day (P) 60-120. Virus AAVlsynGCaMP6f (see, e.g., Reference 14) was injected to both layer 2/3 and layer 5 of the left VI of the mouse cortex, 4-5 weeks prior to the craniotomy surgery. The virus was front-loaded into the injection pipette and injected at a rate of 80 nl/min. The injection sites were at 2.5 mm lateral and 0.3 mm anterior from the lambda, putative monocular region at the left hemisphere. Injections were made at two different depths from the cortical surface, layer 2/3 at 200 μπι-250 μπι and layer 5 at 400 μπι-500 μπι respectively.
[00111] After 4-5 weeks of expression, mice were anesthetized with isoflurane (e.g., 2% by volume, in air for induction and 1% during surgery). Before surgery, dexamethasone sodium phosphate (e.g., 2 mg per kg of body weight; to prevent cerebral edema) and bupivacaine (e.g., 5 mg/ml) were administered subcutaneously, and enrofloxacin (e.g., 4.47 mg per kg) antibiotics and an anti-inflammatory, carprofen (e.g., 5 mg per kg) were administered intraperitoneally. A circular craniotomy (e.g., 2 mm in diameter) was made above the injection site using a dental drill and the dura mater was removed (See, e.g., Reference 36). 1.5% agarose was placed over the craniotomy and a 3-mm circular glass coverslip (e.g., Warner instruments) was placed and sealed using a cyanoacrylate adhesive. A titanium head plate with a 4 mm by 3.5 mm imaging well was attached to the skull using dental cement. The imaging experiments were performed 1- 14 days after the chronic window implantation. During the imaging, the mouse was awake and moved freely on a circular treadmill with its head fixed. Over 40 total imaging sessions across 17 mice were performed. [00112] The shrimp used in the structural imaging were artemia nauplii (e.g., Brine Shrimp Direct) in hydra medium (e.g., with composition of ImM CaCl2*H20, O. lmM MgCl2*6H20, 30μΜ KN03, 0.5mM NaHC03, 80μΜ MgS04). During imaging, the oxygen concentration in the medium was reduced by limiting perfusion, which can lead to reduced motion of the shrimp.
Exemplary Two-photon SLM Laser Scanning Microscope
[00113] The setup of the exemplary two-photon SLM laser scanning microscope is illustrated in Figure 1 A. The laser source can be a pulsed Ti: Sapphire laser (e.g., Coherent Mira HP) which can be tuned to 940 nm with a maximum output power of approximately 1.4 W (e.g., approximately 140 fs pulse width, 80 MHz repetition rate). The output laser beam can be directed through a Pockets cell (e.g., ConOptics EO350-160-BK) to control the intensity on the sample. A broadband λ/2 waveplate (e.g., Thorlabs AHWP05M-980) was located after the EO modulator to rotate the laser polarization so that it can be parallel with the active axis of the spatial light modulator (e.g., Meadowlark Optics, HSP512-1064, 7.68 χ 7.68 mm2 active area, 512 x 512 pixels) located further downstream. The laser beam can be expanded by a 1 : 1.5 telescope (e.g., fl=50 mm, f2=75 mm, Thorlabs plano-convex lenses, "B" coated) to fill the active area of the SLM. The light incident angle to the SLM can be approximately 3.5o. The reflected beam from the SLM can then be scaled by a 4: 1 telescope (e.g., f3=400 mm, f4=100 mm, Thorlabs Achromatic doublets lenses, "B" coated), and imaged onto a set of galvanometer mirrors (e.g., Cambridge 6215HM40B). The galvanometer mirrors can be located conjugate to the microscope objective pupil of a modified Olympus BX-51 microscope by use of an Olympus pupil transfer lens (e.g., f5=50 mm) and Olympus tube lens (e.g., fTL=180 mm). An Olympus 25X NA 1.05
[00114] XLPlan N objective can be used for the imaging. Other objectives also perform well in the exemplary system. The SLM surface can be imaged afocally to the back pupil of the objective. The fluorescent signal from the sample can be detected with a photo-multiplier tube ("PMT", Hamamatsu H7422P-40) located above the microscope, and followed by a low noise amplifier (Stanford Research Systems SR570). Scanlmage 3.8 {see, e.g., Reference 47) can be used to control the galvanometer mirrors, digitize and store the signal from the amplifier of the PMT. The line scanning can be bidirectional with a single line scan rate of 2 kHz. For a 256 x 200 pixel imaging, the frame rate can be 10 fps. Exemplary Hologram Generation
[00115] Custom software using MATLAB (The MathWorks, Natick, MA) was developed to load and control the phase hologram pattern on the SLM through a PCIe interface (Meadowlark Optics). The SLM was calibrated, and a lookup table that converts the pixel values to voltage driving the liquid crystal of the SLM was generated. At the operation wavelength of 940 nm, the SLM outputs approximately 80 effective phase levels over a 2π phase range, with a relative uniform phase level spacing. {See, e.g., graph shown in Figure 7A).
[00116] To create a 3D beamlet pattern at the sample (e.g., a total of N beamlets, each with a coordinates[xi, yi, zj, i=l,2...N), the phase hologram pattern on the SLM φΑα, v) can be, for example expressed in the following:
Φ(ιι, ν) = phase{∑[l1 Aie2^iXiU+yiU+[z°(u'v)c°(Zi)+z°(u'v)c°(Zi)+z^ (Ml)
[00117] Ai can be the electrical field weighting factor for the individual beamlet. ^(u, v) and Cm(Zi) can be the Zernike polynomials and Zernike coefficients, respectively, which can fulfill the defocusing functionality and can compensate the first-order and second-order spherical aberration due to defocusing. The expressions of Zm (u, v) and Cm(zi ) are shown below. (See, e.g., Reference 5). A 2D coordinate calibration between the SLM phase hologram, and thus the projected beamlet pattern at the sample through Eq. M and the PMT image can be carried out on a pollen grain slide. By steering the beamlet to different positions with the SLM and recording the PMT image, an affine transformation can be extracted to map the coordinates between the PMT image and the image plane at the sample, and thus the SLM phase hologram through Eq. Ml . For the axial defocusing, the defocusing length set in the SLM phase hologram can be matched with the actual defocusing length by adjusting the apparent "effective N.A." the Zernicke coefficients, after calibration following the procedure described in (See, e.g., Reference 49). This can be done mainly for convenience, and it can be noted that this can change very little over the full axial range of the SLM (e.g., a range of 0.43-0.48). In multiplane imaging, the field weighting factor Ai in Eq. Ml can determine the power ratio, of different imaging planes. It can be important to note that this coefficient may not be simply the desired power ratio rather it can affect the interference between the different fields, which in turn can depend on the relative phase displacements of the beams. Operationally, either the parameter can be adjusted empirically to achieve similar fluorescent signals from different imaging planes, or alternately, the expected power ratio from first principles can be calculated, considering the depths of each plane, nominal scattering length of light in the tissue, and perform numerical beam propagation of the electric field {See, e.g., Reference 54).
[00118] Visual stimuli were generated using MATLAB (e.g., The MathWorks, Natick, MA) and the Psychophysics Toolbox {see, e.g., Reference 12) and displayed on a liquid crystal display ("LCD") monitor (e.g., 19-inch, 60-Hz refresh rate) position 28 cm from the right eye, at approximately 45o to the long axis of the animal. Each visual stimuli session consisted of 8 different trials, each trial with a 3 s drifting square gratings (e.g., 100% contrast, 0.035 cycles per degree, two cycles per second) display, followed by 5 s of mean luminescence gray screen. 8 drifting directions (e.g., separated by 45 degrees) were presented in random order in the 8 trials in each session. 17 sessions were recorded continuously (e.g., 1088 s). A silicon photodetector (e.g., Thorlabs DET 36 A) can be placed at the corner of the LCD to monitor the visual stimulation display. The detector output was recorded simultaneously with the imaging data at Scanlmage 3.8. The neuronal responses to the visual stimulation can then be extracted and analyzed in the subsequent image processing.
Exemplary Image Analysis and Source Separation Procedure
[00119] The raw images can first be processed to correct brain motion artifacts using an exemplary pyramid procedure {see, e.g., Reference 60), and then analyzed using a constrained non-negative matrix factorization procedure, coded in MATLAB (e.g., The MathWorks, Natick, MA). The core of the CNMF can be that spatiotemporal fluorescence signals F from the whole recording can be expressed as a product of two matrices: (i)a spatial matrix A that encodes the location of each ROI and (ii) a temporal matrix C that characterizes the fluorescent signal of each ROI as well as the background B, expressed as F=AC+B.
[00120] This can be solved as a convex optimization problem, and the assumption that both matrices can be sparse such that they can promote localized spatiotemporal footprints. As shown in the graph of Figure 6B, the individual single plane recordings 615 can first be analyzed, {see, e.g., Reference 46), and the characteristics of each ROI can then be used as initial estimates for the sources in the analysis of the multiplane recording (e.g., dual plane recording 620).
Exemplary Evaluation of CNMF and Comparison with ICA [00121] To evaluate the signals extracted from CNMF, they can be compared with the signal extracted in a conventional manner. To obtain the latter signal, the spatial pixels of each ROI, which do not overlap with other ROIs, can be extracted. Raw temporal signals from these pixels can be averaged with a unitary weight, followed by a subtraction of the background baseline obtained from CNMF. This can be termed as non-overlapped ("NOL") signal. This can be what the conventional fluorescent extraction procedure would do except that the background baseline can be further subtracted.
[00122] The signals extracted from CNMF can be compared with the NOL signals in two aspects: similarity and signal-to-noise ratio ("SNR"). The SNR can be calculated based on the signal's power spectral density.
[00123] ICA was also performed to analyze the data, with the software written in Matlab {See, e.g., Reference 39). The motion-corrected image recording stack can first be normalized, followed by principal component analysis ("PCA") for dimensionality reduction and noise removal. ICA can then be applied to extract the spatio-temporal information of each
independent sources (e.g., ROI).
Exemplary Analysis Of The Cell Orientation Selectivity Of The Drifting Grating Visual Stimulation
[00124] To analyze the orientation and direction selectivity of the ROIs in response to the drifting grating visual stimulation, the total number of events can be counted during the visual stimulation period in each session for all 8 different grating angles. These event numbers can be into a vector space {See, e.g., Reference 38). The direction and magnitude of their vector sum can represent the orientation selectivity and the orientation index. With NviSUaisession visual stimulation sessions, NViSUaisession vectors can be obtained. Hotelling's T2-test can be used to calculate whether these vectors can be significantly different from 0, for example, whether the ROI has a strong orientation selectivity. Only ROIs with their vectors significantly different from 0 (e.g., less than 0.25 probability that null 0 can be true) in the single plane recording can be selected, and their orientation selectivity can be calculated by averaging the NViSUaisession vectors and extracting the angle (e.g.,
Figure imgf000034_0001
for the experiment shown in Figures 6A- 6D2). [00125] The maximal lateral displacement of the incoming beam can be controlled by the familiar grating equation {See, e.g., References 46 and 65):
τ λ = d sin(0) (SI) where m can be the diffraction order, λ the wavelength of the light, d, the feature spacing, d, the angle of deflection. Only the first order m = 1, and the imaging wavelength can be 940 nm. For a pixelated device, the smallest periodic feature can consist of two pixels, each of which can be 15 μπι. Thus d = 30μπι. The maximum SLM induced deflection angle can be related to the maximum lateral displacement from the center of the FOV in the sample plane, rmax, through the following relations:
r - fob) ' tan(#ofc; ) (S2)
Figure imgf000035_0001
MsLM M 1 1 scope
[00126] For the exemplary system, with a 25X objective, this can correspond to for example:
fobj = ftubeiens/Mobj = 180mm/25 = 7.2mm
MSLM = 400/100 = 4.0, Mscope = 50/180 = 0.278
A
i¾¾* = sirT1 1 ((-——) 1 == 1.8° r = 252 τη
[00127] For single point targeting, the efficiency of redirection can depend on the number of phase levels available on the SLM. The exemplary device was measured, and approximately 80 usable phase levels between 0 and 2π were found. (See, e.g., Figure 7A). With the simplifying assumption that these levels can be uniformly spaced between 0 and 2π, the first order diffractive efficiency of such a device can be given by for example:
*
Figure imgf000035_0002
where N can be the total number of levels. (See e.g., Reference 42). This can imply very high efficiencies for holograms with large features (e.g., read spanning many pixels), which can result in small positional displacements in the sample (e.g., Fourier) plane. For larger sample plane displacements, the beneficial phase features can get smaller and smaller, and the finite sampling from the pixelated SLM can reduce the available phase levels. In the extreme case, there can be aliasing, which can strongly degrade the performance; (i)that limit can be avoided for all holograms used. Four pixels can be considered across a Δ2π ramp to be the limit of efficient power redirection. The deflection vs. theoretical efficiency curve is shown in Figure 7D, along with the measured relative SLM power deflection efficiency on the sample {See, e.g., Figures 7C and 7D). For pixelated SLMs, as shown in the graph of Figure 7D which illustrates the calculated power 705 and the measured power 710, with quantized phase levels, the theoretical scalar beam redirection efficiency can be reduced for deflections with a non-integral number of pixels per phase ramp, but this effect can be small, as can be seen in the measured power on sample.
[00128] At rmax, the maximum possible lateral deflection, the pattern can have only two phase levels, which can lower the maximum diffraction efficiency, and additionally, can result in a symmetric phase pattern. Under these conditions, the phase "grating" may not be blazed, but instead can split the beam equally to the +1 and -1 orders; there can be two spots, mirrored across the nominal center of the FOV. This may not be a problem for dual lateral region imaging, where such a split can be in fact advantageous. However, if the desired effect can be to have a single targeted point on the sample, it can be best to restrict the lateral targeting to regions that can be address by phase gratings that span at least four pixels (e.g., levels). With four pixels blaze and two photon excitation, the excitation ratio of the +1 to -1 order can be ~ 20: 1 ; for five pixels, approximately 50: 1 ; and for eight pixels, approximately 350: 1.
[00129] Similar relationships hold for axial displacement. In such exemplary case, the SLM can be used to impart a spherical curvature to the wavefront that can shift the focus before or beyond the nominal focal plane. For simplicity, only primary defocus can be considered here, neglecting higher order aberration terms.
[00130] For a phase wrapped device, this can correspond to a Fresnel phase profile. (See e.g., Reference 42):
Pp = 2pf
[00131] With p, the normalized pupil radius, p, the Fresnel zone order, f, the focal length and λ the wavelength. Each successive zone can correspond to adding one wave of defocus at that point on the pupil, which can also correspond to a 2π phase change. The "strongest" lens allowed without aliasing on the exemplary actual device can correspond to a two pixel spacing on the outermost Fresnel zone. For large p, it can be found that the relative spacing between zones can be, for example:
Δρρ = 2/λ/ρρ = / = 2 d2 ■ ^j- = 123 mm,
with d the pixel size, and N the number of pixels on the device across the shortest dimension.
[00132] The sign of f can be made positive or negative simply by reversing the phase. With this magnitude for the focal length on the SLM, and the exemplary intermediate optical scaling onto the objective back aperture (e.g., effective excitation NA approximately 0.45), this lens power can correspond to axial displacements of ±300 μιη under the objective. Any larger displacement can result in aliasing on the SLM, which can both reduce diffraction efficiency, as well as create secondary foci. For larger axial displacements, without aliasing, an SLM of the same overall all size with smaller pixels, or a larger overall SLM with more pixels, or both, would be beneficial.
[00133] For a physically larger SLM, there can be demagnification to scale the SLM to the appropriate size on the back aperture of the objective; thus the image of the SLM on the back aperture can have higher pixel density. The exemplary preference can be to have more pixels rather than smaller pixels (e.g., smaller pixels typically have lower fill factors, and larger pixels can have increased power handling, as the overall power density on the physical device can be decreased with increasing SLM area). The fill factor can be important for two reasons. The first can be the overall power transmission to the diffracted beam, which can be directly proportional to the fill factor. The second reason can be more subtle.
[00134] For a given diffraction pattern, having fill factors less than one can alter the diffraction period cell length compared to the structural cell length, which can broaden the far-field envelope that governs the relative efficiencies between different diffraction orders. As a result, power can be transferred from the desired first order into higher orders, may not be available for excitation and can generate spurious background signals. (See e.g., Reference 42).
Exemplary Chromatic Effects
[00135] SLMs can be diffractive optics, so they can be chromatic devices where beam redirection can depend on wavelength, (See, e.g., Equation (SI)). Thus any non-monochromatic source (e.g. femtosecond pulsed laser) can exhibit chromatic effects. The relationship connecting bandwidth to pulse width can be ΔνΔΐ > K, with K=0.315 for the hyperbolic secant- squared shaped pulses from most Ti- Sapphire lasers. (See, e.g., Reference26 and 68). The exemplary laser produced transform limited pulses of 140 fs, which means the FWHM of the nominally 940 nm beam can be 6.5 nm. Considering the blue and red edge of this spectral profile, for displacements in the x-y plane, there can be a shift in lateral position with across the spectrum. For the largest lateral displacements used, the relative shift for these components can be less than 1.5 μπι. Nonetheless, because the nominal diffraction limited spot can be of comparable size, this can lead to a minor loss in lateral resolution, as well as a drop in peak intensity at the margins of the FOV. The reduced intensity can be compensated with appropriate weighting of the targets during hologram calculation (see, e.g., Figures 7E and 7F), but the "blur" cannot.
[00136] In the exemplary analysis for lateral deflections, variations in phase efficiency from shifts in the optimal phase depth as a function of wavelength were ignored. These can result from both a wavelength dependent retardation in the liquid crystal itself; the liquid crystals used in the exemplary SLM have only very minor variation in refractive index as a function of wavelength around the exemplary chosen center (e.g., private communication, Meadowlark).
[00137] For axial focusing, this effect can also be ignored. The relative efficiency of a diffractive lens as a function of wavelength can be (see e.g., Reference 42), for example:
_ sin [π( m)] . ^ _ Γ η(Α) i i ^ ^ diffraction order. For the bandwidth of the exemplary laser, η(λ) ~ constant, = = - = [0.995 < < 1] η = [0.998 < η≤ 1] for lenses operating in the first order, or a change in efficiency of 0.2%. The focal length can change with the wavelength, and axial color can be similar, and proportional to 7-. For the effective excitation NA used for functional imaging, NA approximately 0.45, the FWHM of the axial two photon point spread function can be approximately
Figure imgf000038_0001
e.g., References 64 and 69). For the largest displacement used, approximately 300 μπι, the axial color across the FWHM of the laser spectrum can be approximately 2μπι, which represents a total shift of 25% of the axial PSF. This can have a negligible effect on the axial resolution, and power delivery for somatic imaging. Assuming the excitation NA can be changed, the relative effect of the axial color can be invariant with NA, though the absolute sizes of everything scale as NA2.
[00138] Figures 8A and 8B are shows ROI contour and calcium signals according to an exemplary embodiment of the present disclosure. For example, as illustrated in Figure 8 A the ROI contour and calcium signal extracted by CNMF. Element 805 is the contour overlaid on the temporal standard deviation image of a single plane recording of mouse VI at depth of 500 μιη for cortical surface. Traces 810 are the normalized AF/F traces with a scale bar of 50 μιη. Clean, cell like features can be selected, and the great majority of traces have signals with high SNR, and that appear like familiar Ca++ activity traces. Figure 8B is similar to Figure 8A, with ROI contour and calcium signal extracted by ICA. The ROIs can be less cell- like, and a large number of extracted traces show poor SNR, with no features recognizable as typical activity traces.
Exemplary Power Directed to Each Layer
[00139] The mouse brain can be highly scattering, and there can be an exponential loss of ballistic photons with increasing depth. The nominal intensity in the focus as a function of depth can be given by Id = /0 exp [-^J, where /0 can be the incident intensity at the surface of the brain, d, the depth, and 4, the characteristic scattering length. For 940 nm, the scattering length can be approximately 185 μιη (See, e.g., References 27 and 70). Below the saturation limit, two- photon fluorescence signal can scale as the intensity squared. If it can be assumed that the expression of the indicator can be the same in the desired layers for imaging, the relative power needed to be delivered to each layer to give equivalent signal can be estimated as, for example:
F1 oc σ I /01
Figure imgf000039_0001
For the experiment shown in Figures 3 A-3I, this ratio can be as follows: n -2(500 - 170)
= exp = 0.168,
02 185
02 * 6 I01 [00140] This can be combined with the measured displacement efficiency curve (see, e.g., graph shown in Figure 7B) to flexibly set the appropriate weights for each plane in the hologram computation.
[00141] Figures 10A-10O show the Axial Three Plane in-vivo Functional Imaging of Mouse VI at Layer 2/3, 4 and 5, and Extended Simultaneous Imaging Depth over 500 μιη. For example, the top panels of Figures 1 OA- IOC show, temporal standard deviation images of the sequential single plane recording of mouse VI at depth of 170 μιη (e.g., layer 2/3), 350 μιη (e.g., layer 4) and 500 μιη (e.g., layer 5) from a pial surface. The bottom panels of Figures lOA-lOC show ROI contours overlaid on the top panel. The images are false-colored. Scale bar, 50 μιη. Figure 10D shows an image of the arithmetic sum of Figures 1 OA- IOC. The top panel of Figure 10E shows the temporal standard deviation imaging of the simultaneous three-plane recording of the planes shown in Figures 1 OA- IOC. The bottom panel of Figure 10E shows an exemplary image of overlaid ROI contours from the three plane. Figure 10F shows a trace diagram with represented extracted AF/F traces of the selected ROIs from the three planes (e.g., trace 1005, layer 2/3, 15 ROIs out of 58; trace 1010, layer 4, 15 ROIs out of 65; trace 1015, layer 5, 15 ROIs out of 95), from the sequential single plane recording.
[00142] Figure 10G shows an exemplary trace diagram of extracted AF/F traces of the same ROIs shown in Figure 10F, from the simultaneous three-plane recording. Figure 10H shows an exemplary diagram of the source separation of the fluorescent signal from spatially overlapped ROIs in the three plane imaging. The contours of the overlapped ROIs are plotted with their ROI ID. The calcium traces are normalized AF/F traces. For each ROI, the signal is extracted using NOL shown by element 1020, and CNMF by element 1025. The corresponding ROI contours results from these methods are shown next to their AF/F traces. Using the ROI contour in the CNMF but with uniformed pixel weighting and without unmixing treatment, the extracted AF/F trace 1030 is plotted, superimposed onto the traces extracted from CNMF.
[00143] Figure 101 shows an exemplary diagram of the simultaneous three-plane imaging extending the imaging depth over 500 μιη. Figure 10J shows an exemplary overlaid temporal standard deviation image of the sequential single plane recording of mouse VI at depth of 130 μιη (e.g., element 1035), 430 μιη (e.g., element 1040) and 640 μιη (e.g., element 1045) from pial surface of a mouse VI with a scale bar of 50 μιη. Figure 10K shows an exemplary temporal standard deviation imaging of the simultaneous three-plane recording of the three planes. Figure 10L shows an exemplary image of the overlaid ROI contours from the three plane on Figure 10 J, while Figure 10M shows the extracted ROI contours. Figure ION shows represented extracted AF/F traces of the selected ROIs from the three planes (e.g., trace 1045, layer 2/3; trace 1050, layer 4; trace 1055, layer 5), from the sequential single plane recording. Figure 10O shows the extracted AF/F traces of the same ROIs shown in Figure 10N, from the simultaneous three-plane recording.
[00144] Figure 11 A shows an exemplary diagram of SLM switching between two sets of dual plane imaging on mouse VI . State 1 is the dual plane for depth of 210 μιη and 330 μιη from cortical surface, and state 2 is the dual plane for depth of 110 μιη and 270 μιη from pial surface. Imaging frame rate is 10 fps. The SLM switching happens at the middle and at the end of each frame. The zoom-in-view of the switching region shows that the switching time between the two state is less than 2.5 ms.
[00145] Figure 1 IB shows a graph of the switching time between State 1 and State 2, measure from fluorescent signal emitted from Rhodamine 6G. The switching time between different states is less than 3 ms. The black indicator 1105 indicates when the switching starts. Figure l lC shows a diagram of a volumetric imaging procedure using time-multiplexed three-plane imaging. In this example, the normal frame rate is 30 fps. With time-multiplexing between different sets of three-plane imaging, the volumetric imaging rate is 10 vol/s.
[00146] Figure 13 shows an exemplary method 1200 according to an exemplary embodiment of the present disclosure. For example, at procedure 1205, a radiation can be generated using, for example, a laser source. At procedure 1210, the generated radiation can be provided to a SLM, which can split the radiation into a plurality of radiation beamlets at procedure 1215. At procedure 1220, the SLM can independently dynamically control the radiation beamlets and provide them to a galvanometer. At procedure 1225, the beamlets can be directed to the sample using the galvanometer (e.g., to different areas and/or to different planes), which can be used to sweep the beamlets across the sample at or 1230. At procedure 1235, a resultant radiation can be received from the sample that can be based on the beamlets. At procedure 1240, a plurality of images can be generated based on the resultant radiation. At procedure 1245, the images can be interleaved, and one or more multiplane images can be generated at procedure 1250. At procedure 1255, brain motion artifacts can be corrected for in the one or more multiplane images. Additionally, or in the alternative, brain notion artifacts can be corrected for in the plurality of images before the one or more multiplane images are generated.
[00147] Figure 13 shows a block diagram of an exemplary embodiment of a system according to the present disclosure. For example, exemplary procedures in accordance with the present disclosure described herein can be performed by a processing arrangement and/or a computing arrangement 1302. Such processing/computing arrangement 1302 can be, for example entirely or a part of, or include, but not limited to, a computer/processor 1304 that can include, for example one or more microprocessors, and use instructions stored on a computer-accessible medium (e.g., RAM, ROM, hard drive, or other storage device).
[00148] As shown in Figure 13, for example a computer-accessible medium 1306 (e.g., as described herein above, a storage device such as a hard disk, floppy disk, memory stick, CD- ROM, RAM, ROM, etc., or a collection thereof) can be provided (e.g., in communication with the processing arrangement 1302). The computer-accessible medium 1306 can contain executable instructions 1308 thereon. In addition or alternatively, a storage arrangement 1310 can be provided separately from the computer-accessible medium 1306, which can provide the instructions to the processing arrangement 1302 so as to configure the processing arrangement to execute certain exemplary procedures, processes and methods, as described herein above, for example.
[00149] Further, the exemplary processing arrangement 1302 can be provided with or include an input/output arrangement 1314, which can include, for example a wired network, a wireless network, the internet, an intranet, a data collection probe, a sensor, etc. As shown in Figure 13, the exemplary processing arrangement 1302 can be in communication with an exemplary display arrangement 1312, which, according to certain exemplary embodiments of the present disclosure, can be a touch-screen configured for inputting information to the processing arrangement in addition to outputting information from the processing arrangement, for example. Further, the exemplary display 1312 and/or a storage arrangement 1310 can be used to display and/or store data in a user-accessible format and/or user-readable format.
[00150] The foregoing merely illustrates the principles of the disclosure. Various
modifications and alterations to the described embodiments will be apparent to those skilled in the art in view of the teachings herein. It will thus be appreciated that those skilled in the art will be able to devise numerous systems, arrangements, and procedures which, although not explicitly shown or described herein, embody the principles of the disclosure and can be thus within the spirit and scope of the disclosure. Various different exemplary embodiments can be used together with one another, as well as interchangeably therewith, as should be understood by those having ordinary skill in the art. In addition, certain terms used in the present disclosure, including the specification, drawings and claims thereof, can be used synonymously in certain instances, including, but not limited to, for example, data and information. It should be understood that, while these words, and/or other words that can be synonymous to one another, can be used synonymously herein, that there can be instances when such words can be intended to not be used synonymously. Further, to the extent that the prior art knowledge has not been explicitly incorporated by Reference herein above, it is explicitly incorporated herein in its entirety. All publications Referenced are incorporated herein by Reference in their entireties.
EXEMPLARY REFERENCES
[00151] The following References are hereby incorporated by Reference in their entirety.
1. Ahrens, M.B., Orger, M.B., Robson, D.N., Li, J.M., and Keller, P.J. (2013). Whole-brain functional imaging at cellular resolution using light-sheet microscopy. Nature Methods 10, 413-+.
2. Alivisatos, A.P., Andrews, A.M., Boyden, E.S., Chun, M., Church, G.M., Deisseroth, K., Donoghue, TP., Fraser, S.E., Lippincott-Schwartz, J., Looger, L.L., et al. (2013a).
Nanotools for Neuroscience and Brain Activity Mapping. Acs Nano 7, 1850-1866.
3. Alivisatos, A.P., Chun, M., Church, G.M., Deisseroth, K., Donoghue, J.P., Greenspan, R.J., McEuen, P.L., Roukes, M.L., Sejnowski, T.T, Weiss, P.S., et al. (2013b). The Brain
Activity Map. Science 339, 1284-1285. Alivisatos, A.P., Chun, M., Church, G.M.,
Greenspan, R.J., Roukes, M.L., and Yuste, R. (2012). The Brain Activity Map Project and the Challenge of Functional Connectomics. Neuron 74, 970-974.
4. Andresen, V., Egner, A., and Hell, S.W. (2001). Time-multiplexed multifocal multiphoton microscope. Optics Letters 26, 75-77.
5. Anselmi, F., Ventalon, C, Begue, A., Ogden, D., and Emiliani, V. (2011). Three- dimensional imaging and photostimulation by remote-focusing and holographic light patterning. Proceedings of the National Academy of Sciences of the United States of America 108, 19504-19509.
6. Bahlmann, K., So, P.T.C., Kirber, M., Reich, R., Kosicki, B., McGonagle, W., and Bellve, K.
(2007). Multifocal multiphoton microscopy (MMM) at a frame rate beyond 600 Hz. Optics Express 15, 10991-10998. Beaurepaire, E., and Mertz, J. (2002). Epifluorescence collection in two-photon microscopy. Applied Optics 41, 5376-5382.
7. Beckmann, C.F., and Smith, S.A. (2004). Probabilistic independent component analysis for functional magnetic resonance imaging. Ieee Transactions on Medical Imaging 23, 137-152.
8. Bewersdorf, J., Pick, R., and Hell, S.W. (1998). Multifocal multiphoton microscopy. Optics Letters 23, 655-657.
9. Botcherby, E.J., Juskaitis, R., Booth, M.J., and Wilson, T. (2008). An optical technique for remote focusing in microscopy. Optics Communications 281, 880-887.
10. Botcherby, E.J., Juskaitis, R., and Wilson, T. (2006). Scanning two photon fluorescence microscopy with extended depth of field. Optics Communications 268, 253-260. 11. Botcherby, E.J., Smith, C.W., Kohl, M.M., Debarre, D., Booth, M.J., Juskaitis, R., Paulsen, O., and Wilson, T. (2012). Aberration-free three-dimensional multiphoton imaging of neuronal activity at kHz rates. Proceedings of the National Academy of Sciences of the United States of America 109, 2919-2924. Bowsher, J.E., Johnson, V.E., Turkington, T.G., Jaszczak, R.J., Floyd, C.E., and Coleman, R.E. (1996). Bayesian reconstruction and use of anatomical a Priori information for emission tomography. Ieee Transactions on Medical Imaging 15, 673-686.
12. Brainard, D.H. (1997). The psychophysics toolbox. Spatial Vision 10, 433-436.
13. Cha, J.W., Singh, V.R., Kim, K.H., Subramanian, J., Peng, Q., Yu, H., Nedivi, E., and So, P.T.C. (2014). Reassignment of Scattered Emission Photons in Multifocal Multiphoton
Microscopy. Scientific Reports 4. Chen, G.-H., Tang, J., and Leng, S. (2008). Prior image constrained compressed sensing (PICCS): A method to accurately reconstruct dynamic CT images from highly undersampled projection data sets. Medical Physics 35, 660-663.
14. Chen, T.W., Wardill, T.J., Sun, Y., Pulver, S.R., Renninger, S.L., Baohan, A., Schreiter, E.R., Kerr, R.A., Orger, M.B., Jayaraman, V., et al. (2013). Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature 499, 295-+.
15. Cheng, A., Goncalves, J.T., Golshani, P., Arisaka, K., and Portera-Cailliau, C. (2011).
Simultaneous two- photon calcium imaging at different depths with spatiotemporal multiplexing. Nature Methods 8, 139- U158.
16. Ducros, M., Houssen, Y.G., Bradley, J., de Sars, V., and Charpak, S. (2013). Encoded multisite two-photon microscopy. Proceedings of the National Academy of Sciences of the United States of America 110, 13138-13143.
17. Dunn, A.K. (2014). Optical Properties of Neural Tissue. In Optical Imaging of Neocortical 18. Dynamics, B.Weber, and F. Helmchen, eds., pp. 33-51.
19. Egner, A., and Hell, S.W. (2000). Time multiplexing and parallelization in multifocal multiphoton microscopy. Journal of the Optical Society of America a-Optics Image Science and Vision 17, 1192-1201. Fahrbach, F.O., Gurchenkov, V., Alessandri, K., Nassoy, P., and Rohrbach, A. (2013). Light-sheet microscopy in thick media using scanned Bessel beams and two-photon fluorescence excitation. Optics Express 21, 13824-13839.
20. Fittinghoff, D.N., Wiseman, P.W., and Squier, J.A. (2000). Widefield multiphoton and temporally decorrelated multifocal multiphoton microscopy. Optics Express 7, 273-279. 21. Fricke, M., and Nielsen, T. (2005). Two-dimensional imaging without scanning by multifocal multiphoton microscopy. Applied Optics 44, 2984-2988.
22. Golan, L., Reutsky, I, Farah, N., and Shoham, S. (2009). Design and characteristics of holographic neural photo-stimulation systems. Journal of Neural Engineering 6.
23. Grewe, B.F., Langer, D., Kasper, H., Kampa, B.M., and Helmchen, F. (2010). High-speed in vivo calcium imaging reveals neuronal network activity with near-millisecond precision. Nature Methods 7, 399-U391. Grewe, B.F., Voigt, F.F., van 't Hoff, M, and Helmchen, F. (2011). Fast two-layer two-photon imaging of neuronal cell populations using an electrically tunable lens. Biomedical optics express 2, 2035-2046. Helmchen, F., and Denk, W. (2005). Deep tissue two-photon microscopy. Nature Methods 2, 932-940. Helmstaedter, M.,
Sakmann, B., and Feldmeyer, D. (2009). Neuronal Correlates of Local, Lateral, and
Translaminar Inhibition with 24. Reference to Cortical Columns. Cerebral Cortex 19, 926- 937.
25. Helmstaedter, M., Staiger, J.F., Sakmann, B., and Feldmeyer, D. (2008). Efficient
recruitment of layer 2/3 interneurons by layer 4 input in single columns of rat somatosensory cortex. Journal of Neuroscience 28, 8273-8284.
26. Hirlimann, C. (2005). Femtosecond Laser Pulses: Principles and Experiments, Vol
Advanced texts in physics, 2nd edn (Springer).
27. Horton, N.G., Wang, K., Kobat, D., Clark, C.G, Wise, F.W., Schaffer, C.B., and Xu, C.
(2013). In vivo three- photon microscopy of subcortical structures within an intact mouse brain. Nature Photonics 7, 205-209. Huberman, A.D., and Niell, CM. (2011). What can mice tell us about how vision works? Trends in Neurosciences 34, 464-473.
28. Insel, T.R., Landis, S.C., and Collins, F.S. (2013). The NIH BRAIN Initiative. Science 340, 687-688.
29. Iyer, V., Hoogland, T.M., and Saggau, P. (2006). Fast functional imaging of single neurons using random- access multiphoton (RAMP) microscopy. Journal of Neurophysiology 95, 535-545.
30. Ji, N., Sato, T.R., and Betzig, E. (2012). Characterization and adaptive optical correction of aberrations during in vivo imaging in the mouse cortex. Proceedings of the National Academy of Sciences of the United States of America 109, 22-27. 31. Katona, G., Szalay, G., Maak, P., Kaszas, A., Veress, M., Hillier, D., Chiovini, B., Vizi, E.S., Roska, B., and Rozsa, B. (2012). Fast two-photon in vivo imaging with three- dimensional random-access scanning in large tissue volumes. Nature Methods 9, 201-208.
32. Kim, K.H., Buehler, C, Bahlmann, K., Ragan, T., Lee, W.-C.A., Nedivi, E., Heffer, EX., Fantini, S., and So, P.T.C. (2007). Multifocal multiphoton microscopy based on multianode photomultiplier tubes. Optics Express 15, 11658-11678.
33. Kobat, D., Horton, N.G., and Xu, C. (2011). In vivo two-photon microscopy to 1.6-mm depth in mouse cortex. Journal of Biomedical Optics 16.
34. Kremer, Y., Leger, J.F., Lapole, R., Honnorat, N., Candela, Y., Dieudonne, S., and
Bourdieu, L. (2008). A spatio-temporally compensated acousto-optic scanner for two- photon microscopy providing large field of view. Optics Express 16, 10066-10076.
35. Li, Y., Lu, H., Cheng, P.-L, Ge, S., Xu, H., Shi, S.-H., and Dan, Y. (2012). Clonally related visual cortical neurons show similar stimulus feature selectivity. Nature 486, 118-U144.
London, M., and Hausser, M. (2005). Dendritic computation. In Annual Review of
Neuroscience, pp. 503-532.
36. Masamizu, Y., Tanaka, Y.R., Tanaka, Y.H., Hira, R., Ohkubo, F., Kitamura, K., Isomura, Y., Okada, T., and Matsuzaki, M. (2014). Two distinct layer-specific dynamics of cortical ensembles during learning of a motor task. Nature Neuroscience 17, 987-994.
37. Matsumoto, N., Okazaki, S., Fukushi, Y., Takamoto, H., Inoue, T., and Terakawa, S.
(2014). An adaptive approach for uniform scanning in multifocal multiphoton microscopy with a spatial light modulator. Optics Express 22, 633-645.
38. Mazurek, M., Kager, M., and Van Hooser, S.D. (2014). Robust quantification of
orientation selectivity and direction selectivity. Frontiers in Neural Circuits 8.
39. Meyer, H.S., Schwarz, D., Wimmer, V.C., Schmitt, A.C., Kerr, J.N.D., Sakmann, B., and Helmstaedter, M. (2011). Inhibitory interneurons in a cortical column form hot zones of inhibition in layers 2 and 5 A. Proceedings of the National Academy of Sciences of the United States of America 108, 16807-16812. Mukamel, E.A., Nimmerjahn, A., and
Schnitzer, M.J. (2009). Automated Analysis of Cellular Signals from Large-Scale Calcium Imaging Data. Neuron 63, 747-760.
40. Niell, CM., and Stryker, M.P. (2008). Highly selective receptive fields in mouse visual cortex. Journal of Neuroscience 28, 7520-7536. 41. Nikolenko, V., Watson, B.O., Araya, R., Woodruff, A., Peterka, D.S., and Yuste, R. (2008). SLM microscopy: scanless two-photon imaging and photostimulation with spatial light modulators. Frontiers in Neural Circuits 2.
42. O' Shea, D.C., Suleski, T.J., Kathman, A.D., and Prather, D.W. (2003). Diffractive Optics:
Design, Fabrication, and Test, Vol TT62 (Bellingham, Washington, USA: SPIE Press).
43. Otsu, Y., Bormuth, V., Wong, J., Mathieu, B., Dugue, G.P., Feltz, A., and Dieudonne, S.
(2008). Optical monitoring of neuronal activity at high frame rate with a digital random- access multiphoton (RAMP) microscope. Journal of Neuroscience Methods 173, 259-270.
44. Packer, A.M., Peterka, D.S., Hirtz, J.J., Prakash, R., Deisseroth, K., and Yuste, R. (2012).
Two-photon optogenetics of dendritic spines and neural circuits. Nature Methods 9, 1202-
U1103.
45. Packer, A.M., Russell, L.E., Dalgleish, H.W.P., and Haeusser, M. (2015).
Simultaneous all-optical manipulation and recording of neural circuit activity with cellular resolution in vivo. Nature Methods 12, 140-U186.
46. Palmer, C. (2014). Diffraction Grating Handbook, 7th edn (THERMO RGL).
Pnevmatikakis, E.A., Soudry, D., Gao, Y., Machado, T.A., Pfau, D., Reardon, T., Mu, Y.,
Lacefield, C, Poskanzer, K.E., Ahrens, M.B., et al. (2015). Simultaneous denoising, deconvolution, and demixing of calcium imaging data. Neuron.
47. Pologruto, T. A., Sabatini, B.L., and Svoboda, K. (2003). Scanlmage: flexible software for operating laser scanning microscopes. Biomedical engineering online 2, 13-13.
48. Quirin, S., Jackson, J., Peterka, D.S., and Yuste, R. (2014). Simultaneous imaging of neural activity in three dimensions. Frontiers in Neural Circuits 8.
49. Quirin, S., Peterka, D.S., and Yuste, R. (2013). Instantaneous three-dimensional sensing using spatial light modulator illumination with extended depth of field imaging. Optics Express 21, 16007-16021.
50. Reddy, G.D., Kelleher, K., Fink, R., and Saggau, P. (2008). Three-dimensional random access multiphoton microscopy for functional imaging of neuronal activity. Nature
Neuroscience 11, 713-720.
51. Rickgauer, J.P., Deisseroth, K., and Tank, D.W. (2014). Simultaneous cellular-resolution optical perturbation and imaging of place cell firing fields. Nature Neuroscience 17, 1816-
1824. 52. Rochefort, N.L., Narushima, M., Grienberger, C, Marandi, N., Hill, D.N., and Konnerth, A. (2011). Development of Direction Selectivity in Mouse Cortical Neurons. Neuron 71, 425- 432.
53. Salome, R., Kremer, Y., Dieudonne, S., Leger, J.F., Krichevsky, O., Wyart, C, Chatenay, D., and Bourdieu, L. (2006). Ultrafast random-access scanning in two-photon microscopy using acousto-optic deflectors. Journal of Neuroscience Methods 154, 161-174.
54. Schmidt, J.D. (2010). Numerical Simulation of Optical Wave Propagation with Examples in MATLAB (Bellingham, Washington 98227-0010 USA: SPIE Press).
55. Shai, A.S., Anastassiou, C.A., Larkum, M.E., and Koch, C. (2015). Physiology of Layer 5 Pyramidal Neurons in Mouse Primary Visual Cortex: Coincidence Detection through
Bursting. PLoS computational biology 11, el004090-el004090.
56. Sheetz, K.E., Hoover, E.E., Carriles, R., Kleinfeld, D., and Squier, J.A. (2008). Advancing multifocal nonlinear microscopy: development and application of a novel multibeam
Yb:KGd(W04)(2) oscillator. Optics Express 16, 17574-17584.
57. Siltanen, S., Kolehmainen, V., Jarvenpaa, S., Kaipio, J.P., Koistinen, P., Lassas, M., Pirttila, J., and Somersalo, E. (2003). Statistical inversion for medical x-ray tomography with few radiographs: I. General theory. Physics in Medicine and Biology 48, 1437-1463.
58. Stirman, J.N., Smith, I.T., Kudenov, M.W., and Smith, S.L. (2014). Wide field-of-view, twin-region two-photon imaging across extended cortical networks.
59. Theriault, G., Cottet, M., Castonguay, A., McCarthy, N., and De Koninck, Y. (2014).
Extended two-photon microscopy in live samples with Bessel beams: steadier focus, faster volume scans, and simpler stereoscopic imaging. Frontiers in Cellular Neuroscience 8. 60. Thevenaz, P., Ruttimann, U.E., and Unser, M. (1998). A pyramid approach to subpixel registration based on intensity. Ieee Transactions on Image Processing 7, 27-41.
61. Vogelstein, J.T., Packer, A.M., Machado, T.A., Sippy, T., Babadi, B., Yuste, R., and
Paninski, L. (2010). Fast Nonnegative Deconvolution for Spike Train Inference From Population Calcium Imaging. Journal of Neurophysiology 104, 3691-3704.
62. Vogelstein, J.T., Watson, B.O., Packer, A.M., Yuste, R., Jedynak, B., and Paninski, L.
(2009). Spike Inference from Calcium Imaging Using Sequential Monte Carlo Methods. Biophysical Journal 97, 636-655. 63. Williams, R.M., Zipfel, W.R., and Webb, W.W. (2001). Multiphoton microscopy in biological research. Current Opinion in Chemical Biology 5, 603-608.
64. Zipfel, W.R., Williams, R.M., and Webb, W.W. (2003). Nonlinear magic: Multiphoton microscopy in the biosciences. Nature Biotechnology 21, 1369-1377.
65. Palmer, C, Diffraction Grating Handbook. 7th ed. 2014: THERMO RGL. 265.
66. O' Shea, D.C., et al., Diffractive Optics: Design, Fabrication, and Test. Tutorial Texts in Optical Engineering. Vol. TT62. 2003, Bellingham, Washington, USA: SPIE Press.
67. Golan, L., et al., Design and characteristics of holographic neural photo-stimulation systems. Journal of Neural Engineering, 2009. 6(6).
68. Hirlimann, C, Femtosecond Laser Pulses: Principles and Experiments. 2nd ed. Vol.
Advanced texts in physics. 2005: Springer.
69. Zipfel, W.R., R.M. Williams, and W.W. Webb, Nonlinear magic: Multiphoton microscopy in the biosciences. Nature Biotechnology, 2003. 21(11): p. 1369-1377.
70. Horton, N.G., et al., In vivo three-photon microscopy of subcortical structures within an intact mouse brain. Nature Photonics, 2013. 7(3): p. 205-209.

Claims

WHAT IS CLAIMED IS:
1. A device, comprising:
at least one radiation source configured to generate at least one first radiation;
at least one spatial light modulator (SLM) arrangement configured to receive the at least one first radiation and generate at least one second radiation based on the at least one first radiation; and
at least one galvanometer configured to receive the at least one second radiation, generate at least one third radiation based on the at least one second radiation, and provide the at least one third radiation to at least one sample.
2. The device of claim 1, wherein the at least one SLM arrangement includes:
a SLM; and
a pre-SLM afocal telescope configured to resize the at least one first radiation to match an area of the SLM.
3. The device of claim 2, wherein the at least one SLM arrangement further includes:
a plurality of folding mirrors configured to redirect the at least one first radiation to the pre-SLM; and
a post SLM afocal telescope configured to resize the at least one second radiation to match a size of an acceptance aperture of the at least one galvanometer.
4. The device of claim 3, wherein the at least one SLM arrangement further includes at least one broadband waveplate located between the pre-SLM afocal telescope and the SLM.
5. The device of claim 4, wherein the at least one broadband waveplate is configured to rotate a polarization of the at least one first radiation to cause the at least one radiation to be parallel with an active axis of the SLM.
6. The device of claim 1, wherein the at least one SLM arrangement is configured to split the at least one first radiation into the radiation beamlets which are the at least one second radiation.
7. The device of claim 6, wherein the at least one SLM arrangement is further configured to independently dynamically control each of the radiation beamlets.
8. The device of claim 6, wherein the at least one SLM arrangement splits the at least one first radiation into the radiation beamlets by imprinting a phase profile across the at least one first radiation.
9. The device of claim 6, wherein the at least one galvanometer is further configured to direct each of the radiation beamlets to a different area of the at least one sample.
10. The device of claim 6, wherein the at least one galvanometer is further configured to direct each of the radiation beamlets to a different plane of the at least one sample
11. The device of claim 1, wherein the at least one radiation source is at least one laser source.
12. The device of claim 1, further comprising at least one pocket cell located between the at least one radiation source and the at least one SLM arrangement.
13. The device of claim 12, wherein the at least one pocket cell is configured to modulate an intensity of the at least one first radiation.
14. The device of claim 1, further comprising a computer processing arrangement configured to generate at least one image of the at least one sample based on at least one fourth radiation received from the at least one sample that is based on the at least one third radiation.
15. The device of claim 10, further comprising a computer processing arrangement configured to generate at least one image of the at least one sample based on a plurality of resultant radiations received from the at least one sample that are based on the radiation beamlets.
16. The device of claim 15, wherein a first number of the resultant radiations is based on a second number of the radiation beamlets.
17. The device of claim 16, wherein the second number of the radiation beamlets is based on a third number of the planes of the at least one sample.
18. The device of claim 17, wherein the computer processing arrangement is further configured to generate a third number of images of the at least one sample based on the resultant radiations.
19. The device of claim 15, wherein the at least one image includes a plurality of images.
20. The device of claim 19, wherein the computer processing arrangement is further configured to generate at least one multiplane image based on the images.
21. The device of claim 20, wherein the at least one multiplane image is generated by interleaving the images into the at least one multiplane image.
22. The device of claim 20, wherein the computer processing arrangement is further configured to correct brain motion artifacts in the images based on a pyramid procedure.
23. A method, comprising:
generating at least one radiation;
providing the at least one radiation to at least one spatial light modulator (SLM) arrangement;
splitting the at least one radiation into a plurality of radiation beamlets using the at least one SLM arrangement; and
directing the radiation beamlets to at least one sample using at least one galvanometer.
24. The method of claim 23, further comprising, with a computer hardware arrangement, generating at least one image of the at least one sample based on a resultant radiation received from the at least one sample that is based on the radiation beamlets.
25. The method of claim 23, wherein the at least one radiation is generated using at least one laser.
26. The method of claim 23, further comprising independently dynamically controlling each of the radiation beamlets using the at least one SLM arrangement.
27. The method of claim 23, wherein the at least one SLM arrangement splits the at least one radiation into the plurality of radiation beamlets by imprinting a phase profile across the at least one radiation.
28. The method of claim 23, wherein the at least one SLM arrangement includes at least one SLM.
29. The method of claim 23, wherein each of the radiation beamlets is directed to a different area of the at least one sample.
30. The method of claim 20, further comprising sweeping each of the radiation beamlets across the respective different area of the at least one sample using the at least one galvanometer.
31. The method of claim 23, wherein each of the radiation beamlets is directed to a different plane of the at least one sample.
32. The method of claim 31, further comprising receiving a plurality of resultant radiations from the at least one sample that are based on the radiation beamlets.
33. The method of claim 32, wherein a first number of the resultant radiations is based on a second number of the radiation beamlets.
34. The method of claim 33, wherein the second number of the radiation beamlets is based on a third number of the planes of the at least one sample.
35. The method of claim 34, further comprising, with a computer hardware arrangement, generating a third number of images of the at least one sample based on the resultant radiations.
36. The method of claim 32, further comprising, with a computer hardware arrangement, generating a plurality of images based on the resultant radiations.
37. The method of claim 36, further comprising, with a computer hardware arrangement, generating at least one multiplane image based on the images.
38. The method of claim 37, wherein the multiplane images are generated by interleaving the images into the at least one multiplane image.
39. The method of claim 37, further comprising correcting brain motion artifacts in the images based on a pyramid procedure.
PCT/US2016/040753 2015-07-01 2016-07-01 System, method and computer-accessbile medium for multi-plane imaging of neural circuits WO2017004555A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/741,435 US10520712B2 (en) 2015-07-01 2016-07-01 System, method and computer-accessible medium for multi-plane imaging of neural circuits
US16/730,344 US20210033836A1 (en) 2015-07-01 2019-12-30 System, method and computer-accessible medium for multi-plane imaging of neural circuits

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201562187595P 2015-07-01 2015-07-01
US62/187,595 2015-07-01

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US15/741,435 A-371-Of-International US10520712B2 (en) 2015-07-01 2016-07-01 System, method and computer-accessible medium for multi-plane imaging of neural circuits
US16/730,344 Continuation US20210033836A1 (en) 2015-07-01 2019-12-30 System, method and computer-accessible medium for multi-plane imaging of neural circuits

Publications (1)

Publication Number Publication Date
WO2017004555A1 true WO2017004555A1 (en) 2017-01-05

Family

ID=57609597

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2016/040753 WO2017004555A1 (en) 2015-07-01 2016-07-01 System, method and computer-accessbile medium for multi-plane imaging of neural circuits

Country Status (2)

Country Link
US (2) US10520712B2 (en)
WO (1) WO2017004555A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108549275A (en) * 2018-03-01 2018-09-18 南昌大学 A kind of control device and control method of single photon compression imaging
WO2018185201A3 (en) * 2017-04-07 2018-12-27 Carl Zeiss Microscopy Gmbh Microscope assembly for capturing and displaying three-dimensional images of a sample
CN110470640A (en) * 2019-08-08 2019-11-19 清华大学 Two-photon mating plate micro imaging method and device
CN110786853A (en) * 2018-08-03 2020-02-14 中国科学院上海生命科学研究院 Mouse head fixing device for magnetic resonance imaging and fixing method thereof

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10976705B2 (en) * 2016-07-28 2021-04-13 Cy Vision Inc. System and method for high-quality speckle-free phase-only computer-generated holographic image projection
JP6864272B2 (en) * 2017-07-31 2021-04-28 日本電気株式会社 Communication equipment, phase modulation type spatial light modulation element, operation control method, and operation control program
US11221476B2 (en) * 2017-10-12 2022-01-11 Howard Hughes Medical Institute High-resolution, real-time imaging with adaptive optics and lattice light sheets
US11703800B2 (en) 2018-03-21 2023-07-18 The Board Of Trustees Of The Leland Stanford Junior University Methods for temporal and spatial multiplexing of spatial light modulators and systems for same
US11698303B2 (en) * 2018-12-28 2023-07-11 Spectral Engines Oy Method and system for analysing a chemical composition of a target using a Fabry-Perot interferometer
US10677730B1 (en) * 2019-02-01 2020-06-09 Apllikate Technologies Llc Fast multiphoton microscope
JP7262340B2 (en) * 2019-07-31 2023-04-21 富士フイルムヘルスケア株式会社 Ultrasonic CT device, image processing device, and image processing program
CN111208531A (en) * 2020-01-19 2020-05-29 中国科学技术大学 Single photon imaging laser radar system based on wide-spectrum light source
KR20230171945A (en) * 2021-04-26 2023-12-21 에이에스엠엘 네델란즈 비.브이. Cleaning methods and associated light source metrology devices

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4831333A (en) * 1986-09-11 1989-05-16 Ltv Aerospace & Defense Co. Laser beam steering apparatus
US20080049232A1 (en) * 2006-08-25 2008-02-28 The General Hospital Coporation Apparatus and methods for enhancing optical coherence tomography imaging using volumetric filtering techniques
US20080084542A1 (en) * 2006-10-06 2008-04-10 Marc Lalley Three-dimensional internal back-projection system and method for using the same
US20090046333A1 (en) * 2007-08-14 2009-02-19 The Arizona Board Of Regents On Behalf Of The University Of Arizona System and Method Using a Voltage Kick-Off to Record a Hologram on a Photorefractive Polymer for 3D Holographic Display and Other Applications
WO2009136189A1 (en) * 2008-05-07 2009-11-12 Ulive Enterprises Limited Microstructuring
US20110233046A1 (en) * 2008-09-25 2011-09-29 The Trustees Of Columbia University In The City Of New York Devices, apparatus and method for providing photostimulation and imaging of structures
US20130057953A1 (en) * 2011-09-02 2013-03-07 Olympus Corporation Nonlinear optical microscope
US20130181143A1 (en) * 2011-07-14 2013-07-18 Howard Hughes Medical Institute Microscopy with adaptive optics
US20140152795A1 (en) * 2012-12-05 2014-06-05 Olympus Corporation Optical apparatus

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4831333A (en) * 1986-09-11 1989-05-16 Ltv Aerospace & Defense Co. Laser beam steering apparatus
US20080049232A1 (en) * 2006-08-25 2008-02-28 The General Hospital Coporation Apparatus and methods for enhancing optical coherence tomography imaging using volumetric filtering techniques
US20080084542A1 (en) * 2006-10-06 2008-04-10 Marc Lalley Three-dimensional internal back-projection system and method for using the same
US20090046333A1 (en) * 2007-08-14 2009-02-19 The Arizona Board Of Regents On Behalf Of The University Of Arizona System and Method Using a Voltage Kick-Off to Record a Hologram on a Photorefractive Polymer for 3D Holographic Display and Other Applications
WO2009136189A1 (en) * 2008-05-07 2009-11-12 Ulive Enterprises Limited Microstructuring
US20110233046A1 (en) * 2008-09-25 2011-09-29 The Trustees Of Columbia University In The City Of New York Devices, apparatus and method for providing photostimulation and imaging of structures
US20130181143A1 (en) * 2011-07-14 2013-07-18 Howard Hughes Medical Institute Microscopy with adaptive optics
US20130057953A1 (en) * 2011-09-02 2013-03-07 Olympus Corporation Nonlinear optical microscope
US20140152795A1 (en) * 2012-12-05 2014-06-05 Olympus Corporation Optical apparatus

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018185201A3 (en) * 2017-04-07 2018-12-27 Carl Zeiss Microscopy Gmbh Microscope assembly for capturing and displaying three-dimensional images of a sample
CN110431465A (en) * 2017-04-07 2019-11-08 卡尔蔡司显微镜有限责任公司 For shooting and presenting the microscopie unit of the 3-D image of sample
JP2020508496A (en) * 2017-04-07 2020-03-19 カール ツァイス マイクロスコピー ゲーエムベーハーCarl Zeiss Microscopy Gmbh Microscope device for capturing and displaying a three-dimensional image of a sample
CN110431465B (en) * 2017-04-07 2021-12-21 卡尔蔡司显微镜有限责任公司 Microscope device for recording and displaying three-dimensional images of a sample
CN108549275A (en) * 2018-03-01 2018-09-18 南昌大学 A kind of control device and control method of single photon compression imaging
CN110786853A (en) * 2018-08-03 2020-02-14 中国科学院上海生命科学研究院 Mouse head fixing device for magnetic resonance imaging and fixing method thereof
CN110786853B (en) * 2018-08-03 2024-02-23 中国科学院脑科学与智能技术卓越创新中心 Mouse head fixing device for magnetic resonance imaging and fixing method thereof
CN110470640A (en) * 2019-08-08 2019-11-19 清华大学 Two-photon mating plate micro imaging method and device
CN110470640B (en) * 2019-08-08 2021-04-27 清华大学 Two-photon optical sheet microimaging method and device

Also Published As

Publication number Publication date
US20210033836A1 (en) 2021-02-04
US20180373009A1 (en) 2018-12-27
US10520712B2 (en) 2019-12-31

Similar Documents

Publication Publication Date Title
US10520712B2 (en) System, method and computer-accessible medium for multi-plane imaging of neural circuits
Yang et al. Simultaneous multi-plane imaging of neural circuits
Yang et al. Simultaneous two-photon imaging and two-photon optogenetics of cortical circuits in three dimensions
Ji et al. Technologies for imaging neural activity in large volumes
Nöbauer et al. Video rate volumetric Ca2+ imaging across cortex using seeded iterative demixing (SID) microscopy
Kim et al. Long-term optical access to an estimated one million neurons in the live mouse cortex
US20210033843A1 (en) Imaging signal extraction apparatus and methods of using same
JP7180964B2 (en) Three-dimensional imaging apparatus and method
WO2015054450A1 (en) Multiview light-sheet microscopy
EP3205254A1 (en) Medical inspection apparatus, such as a microscope or endoscope, using pseudocolors
Schultz et al. Advances in two-photon scanning and scanless microscopy technologies for functional neural circuit imaging
Ricci et al. Removing striping artifacts in light-sheet fluorescence microscopy: a review
Sacconi et al. KHz-rate volumetric voltage imaging of the whole zebrafish heart
EP3814823A1 (en) Line excitation array detection microscopy
CN109952525B (en) Method for scanning a region of interest by scanning a plurality of three-dimensional lines
Huang et al. Optical volumetric brain imaging: speed, depth, and resolution enhancement
US20170212342A1 (en) Optical microscope system for simultaneous observation of spatially distinct regions of interest
Ota et al. Breaking trade-offs: Development of fast, high-resolution, wide-field two-photon microscopes to reveal the computational principles of the brain
Ohayon et al. Deep brain fluorescence imaging with minimally invasive ultra-thin optical fibers
US9696255B2 (en) Image processing method of two-photon structured illumination point scanning microscopy
CN108882831B (en) Optical imaging device
Xue et al. Multiline orthogonal scanning temporal focusing (mosTF) microscopy for scattering reduction in high-speed in vivo brain imaging
Miller et al. Simultaneous Multi-plane Imaging of Neural Circuits.
JP2022119378A (en) Photoelectric conversion device and apparatus
Guo et al. Fourier-domain light-field microscopy

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16818910

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16818910

Country of ref document: EP

Kind code of ref document: A1