Open Access
Add to BibSonomy
Abstract
Besides traditional lens-based imaging techniques, coded aperture imaging (CAI) can also provide target images but without using any optical lenses, therefore it is another solution in imaging applications. Most CAI methods reconstruct target image only from a single-shot coded image using a fixed coding mask; however, the collected partial information inevitably deteriorates the reconstruction quality. Though multi-exposure CAI methods are designed, these existed algorithms can hardly improve reconstruction signal-to-noise ratio (SNR) and spatial resolution simultaneously; additionally, dynamic coding mask display still requires expensive devices and complicated systems. In order to reconstruct target image with both enhanced spatial resolution and SNR but using cost-effective devices and a simple system, we design a noise reduced dynamic synthetic coded aperture imaging camera (NoRDS-CAIC) in this paper. The NoRDS-CAIC only consists of a programmable liquid crystal display (LCD) and an image recorder, and both of them are integrated with a three-dimensional printed shell with the compact size of 19 cm × 15 cm × 16 cm and controlled by our designed software to automatically realize coding mask display, coded image recording and target image reconstruction. When using the NoRDS-CAIC, the optimized coding mask is first sent to the programmable LCD and displayed, then the corresponding coded image is automatically captured using the image recorder. Next, cycle the above procedures to capture enough coded images with previously known coding masks and measured point spread functions (PSFs), and the target image can be finally reconstructed using our designed NoRDS-CAIC decoding algorithm, which is shown with better noise suppression capability and higher reconstruction resolution compared to other classical CAI algorithms. According to the experimental verifications, the NoRDS-CAIC can reach the high resolution of 99.2 µm and the high SNR of 19.43 dB, proving that the designed NoRDS-CAIC can be potentially used for lensless imaging in practical applications.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
As a lensless imaging technique, coded aperture imaging (CAI) [1–7] has been adopted in many fields for its extensive benefits and applications in past decades. Compared to classical single pinhole optical imaging [8–11] with inherent disadvantages as low light intensity and signal-to-noise ratio (SNR), CAI often has much higher light efficiency and SNR because of the adopted pinhole array coding masks [12]. First introduced by Dicke and Ables [1,2], CAI was adopted to address the problem of low light intensity and SNR for X-ray astronomy. Fenimore and Cannon [13–15] developed the pinhole-array-based CAI technique, and further applied it for X-ray imaging in astronomy and medicine. In addition, CAI was also used in visible and infrared light imaging [16–29]. Zomet et al. designed a lensless camera with a controllable aperture [16]. Veeraraghavan group designed FlatCam which can capture high-quality images as classical lens-based cameras do and can even be used in face detection [17–19]. Wang et al. presented a coded pinhole lens imaging system using pinhole array aperture [20]. Moreover, Brady group obtained hyperspectral imaging with various designs of CAI systems [21]. In addition, CAI was also adopted in fluorescence imaging as fluorescence FlatScope proposed by Veeraraghavan group [19], as well as wavefront imaging as wavefront imaging sensors proposed by Veeraraghavan group [22], Heidrich group [23] and Horisaki group [24]. Moreover, Waller group presented a multiplexed phase-space imaging method for three-dimensional (3-D) fluorescence imaging [27] and then designed DiffuserCam with a thin phase diffuser to implement 3-D imaging [28,29]. Among various CAI works, different devices have been used for coding mask generation: some are directly fabricated only acting as static coding masks [18–20]; besides, digital mirror device (DMD) can generate dynamic binary coding masks in extremely fast speed [21]; additionally, spatial light modulator (SLM) [22–26] or programmable LCD [16] is also a choice for dynamic coding mask generation. Moreover, since SLM has the capability in both amplitude and phase modulations, besides its applications in classical CAI, it has also been extended to new type lensless imaging such as coded aperture correlation holography [30–37]. Besides these CAI devices, reconstruction quality also relies on the decoding algorithms. Various decoding methods have been proposed to pursue high-quality target reconstruction, including Richardson–Lucy method [38], correlation method [13–15,25,26] and deconvolution method [13]. However, these methods can still hardly maintain high resolution and SNR simultaneously. In the Richardson–Lucy method, for a complex-structured pinhole array mask, the decoding result deteriorates due to the limited deblurring capability. Furthermore, the ringing effect also occurs with ongoing iterations. While applying the correlation method, except for short wavelength imaging (eg. X-ray or γ-ray imaging), the accurate distribution of the post-processing array can hardly be acquired, thus inducing the noise overlayed reconstructions. Similarly, the decoding noise may dominate the decoding process if an irregular regularization parameter in Wiener filter is adopted in the deconvolution method, which seriously degrades the reconstruction quality. Though the decoding noise can be reduced or even eliminated with denoising techniques like block matching 3D (BM3D) [39,40], both spatial resolution and SNR of decoded image inevitably decrease. Besides, the decoding noise can also be reduced by multi-exposure method [12]. However, image blurring occurs in the multiple decoded image summation, thus causing the resolution degradation.
To reduce the decoding noise existed in CAI as well as to improve the reconstruction spatial resolution, we previously designed a synthetic CAI method [41] which was revealed with both enhanced resolution and SNR. By rotating the pinhole array, the coded images are recorded, and the iterative synthetic CAI decoding algorithm is adopted to reconstruct the target image as well as to reduce the decoding noise relying on Wiener filter. However, it is still time-consuming in data collection since the coded images are recorded along with a pinhole array rotated mechanically, and the mechanical error inevitably deteriorates the reconstruction accuracy. Furthermore, the pinhole array was not optimized thus may influence the coded information collection. These drawbacks not only make the previously proposed synthetic CAI system complicated, but also cause slow speed and low accuracy in image reconstruction.
In order to reconstruct the target image with both enhanced reconstruction resolution and SNR but using cost-effective devices and simple system, we propose a Noise Reduced Dynamic Synthetic Coded Aperture Imaging Camera (NoRDS-CAIC). The NoRDS-CAIC is rather simple only consisting of a programmable LCD and an image recorder integrated using a 3-D printed shell with a compact size of 19 cm × 15 cm × 16 cm. The mechanically rotated pinhole array in our previous work [41] is replaced by a programmable liquid crystal display (LCD), which can load and display coding masks as binary pinhole arrays precisely and rapidly. Moreover, we have also designed the NoRDS-CAIC software for automatic coding mask display, coded image recording and target image reconstruction. Tested by both simulations and experiments, the NoRDS-CAIC is proved with higher reconstruction SNR and spatial resolution compared to other classical CAI approaches. Moreover, it overcomes the drawbacks of the mechanical coding mask rotation to further accelerate the imaging speed and reduce the rotating error. Considering its advantages as compact system, cost-effective configuration, fast recording speed and high reconstruction quality, it is believed that the designed NoRDS-CAIC can be potentially used for lensless imaging in various applications to pursue high-SNR and high resolution imaging.
2. Decoding algorithm of the NoRDS-CAIC
2.1 Design on the NoRDS-CAIC decoding algorithm
In NoRDS-CAIC, our previously proposed synthetic coded aperture imaging algorithm [41] was adopted and improved, which flowchart is shown in Fig. 1. Moreover, the simulation source code is provided in the Appendix. Both the spatial spectra of the guessed coded image Ig(u, v; k) and the recorded coded image I(u, v; k) are represented in Eq. (1) [42], in which O(u, v), Og(u, v; k), H(u, v; k) and N’(u, v; k) representing target, guessed target, point spread function (PSF) and noise in frequency domain, and k donates the number of the captured coded image.
While minimizing this loss function, a revision to the guessed target spatial spectrum Og(u, v) was applied as shown in Eq. (3), in which ξ is a positive constant and works as the regularization parameter which aims to modulate the noise effect, and * means conjugation.
In the NoRDS-CAIC decoding algorithm, the second term of Eq. (3) takes value close to $\frac{{ - {H^\ast }(u, v; k){N^{\bf ^{\prime}}}({u, v; k} )}}{{{{|{H(u, v; k)} |}^2} + \xi }}$ iteratively by updating the kth coding mask, and this term accumulates as $\sum\limits_k {\frac{{ - {H^\ast }(u, v; k){N^{\bf ^{\prime}}}({u, v; k} )}}{{{{|{H(u, v; k)} |}^2} + \xi }}} \approx 0$ after addressing all the coding masks, thus leading the second term of Eq. (5) approaches to zero. Therefore, the problem of extra modulation on the spatial spectra of an object and the noise amplification effect in the deconvolution method can be mitigated. According to the principle of the NoRDS-CAIC decoding algorithm, higher reconstruction quality can be obtained compared to other existed CAI decoding algorithms. In order to verify the performance of the NoRDS-CAIC decoding algorithm, numerical simulations are provided in the following section.
2.2 Verification on the NoRDS-CAIC decoding algorithm
In order to test the capability of the NoRDS-CAIC decoding algorithm, numerical simulations with the Siemens star were first implemented as shown in Fig. 2. According to the NoRDS-CAIC scheme, the distance between the target and the programmable LCD was 125 mm, and that between the programmable LCD and the image recorder was 35 mm. The pixel sizes of the programmable LCD and the image recorder were set as 132.5 µm and 9 µm, respectively. Figure 2(a) shows the target image and its zoomed field of interest (FoI), and Fig. 2(b) lists a coding mask and its corresponding coded image. In order to reconstruct the target image, the deconvolution method relying on a single-shot coded image was used, and the reconstructed image as well as its zoomed FoI is shown in Fig. 2(c) with low reconstruction SNR and resolution. In addition, the Richardson-Lucy method was also implemented with the reconstruction result shown in Fig. 2(d); unfortunately, the result is poor using a random coding mask here, but when using other ring-structured coding masks, the reconstruction quality can be improved [38]. Moreover, the BM3D aided and the multi-exposure based deconvolution methods were also adopted, and their reconstructed target images are listed in Fig. 2(e) and Fig. 2(f) from single coded image and 36 coded images, respectively. Compared to the results in Figs. 2(c) and 2(d), the reconstruction quality of these two methods is obviously improved. However, the reconstructed target image using the NoRDS-CAIC decoding algorithm in Fig. 2(g) has the highest quality among these algorithms and is rather close to the original target in Fig. 2(a).
Fig. 2. Simulation results. (a) Target image. (b) Coding mask and coded image. Reconstructed target images with (c) deconvolution method, (d) Richardson-Lucy method, (e) BM3D aided deconvolution method, (f) multi-exposure method and (g) NoRDS-CAIC decoding method. The inserted images are zoomed FoI. (h) MSE and PSNR of the reconstructed images with increasing iteration numbers using NoRDS-CAIC decoding method. The inserts are zoomed FoI of the reconstructed target images with 1, 5, 10, 20, 30 and 36 iterations, respectively. (i) MSE of the reconstruction results using different decoding methods in various noise levels. (j) MSE of the reconstruction results using NoRDS-CAIC decoding method in different numbers of coding masks and different PSNR of coded images.
According to the reconstruction results listed in Fig. 2(c) to Fig. 2(g), it is clear that the reconstruction resolution of the NoRDS-CAIC decoding algorithm is higher than that of other CAI decoding methods. Additionally, mean square error (MSE) in Eq. (4) and peak signal to noise ratio (PSNR) in Eq. (5) were used to estimate the reconstruction quality and the noise suppression capability, respectively. In Eqs. (4) and (5), o’k(x, y) is the reconstructed image using k coded images, o(x, y) is the distribution of the original image, M × N is image size, and m is the maximal intensity of the decoded image. As shown in Fig. 2(h), the NoRDS-CAIC decoding algorithm is able to obtain a converged result with high PSNR and low MSE. Besides, the inserted images shown in Fig. 2(h) clearly show the resolution improvement with increased iterations, and the PSNR of the reconstructed target image can reach 32.42 dB, which is improved by ∼3.5 times compared to deconvolution method of 9.31 dB. Moreover, the PSNR of Fig. 2(h) is higher than those in Figs. 2(d)–2(f) of 5.94 dB, 12.92 dB and 23.01 dB, respectively. Though Richardson-Lucy and BM3D methods show ability of image denoising with fewer images, spatial resolution of the decoded images inevitably decreases. While the analysis strongly supports that the NoRDS-CAIC decoding algorithm can reconstruct the target image with both enhanced resolution and SNR.
According to the numerical simulations, it is shown that the NoRDS-CAIC decoding algorithm can reconstruct the target image with both enhanced resolution and SNR; and compared to other traditional CAI decoding methods, it can reconstruct the target image with the highest quality and resolution as well as the best noise suppression capability. Moreover, in order to balance the imaging quality and the imaging efficiency, 30 coded images are required when the SNR of captured coded images is no less than 20 dB. According to the proposed numerical simulations, it is revealed that the updated version of our previously designed synthetic CAI decoding method can be used in NoRDS-CAIC for various imaging applications.
3. Design of the NoRDS-CAIC
3.1 Hardware design and assemble of the NoRDS-CAIC
Figure 3(a) shows the scheme of the NoRDS-CAIC, which consists of a programmable LCD and an image recorder. The programmable LCD (Raspberry Pi HDMI LCD) is an electronic integrated display with a panel size of 107 mm × 68.5 mm (5 inch, 800 × 480 pixels with each pixel size of 132.5 µm) sufficient to cover the entire CCD camera target. There is black matrix on each pixel of LCD and the effective aperture ratio is limited to ∼70%. Though the programmable LCD allows different transmission levels for each pixel, in this NoRDS-CAIC design, only two gray levels were used representing the transparent and opaque states, and their contrast was set to be maximum to maintain the maximal contrast between these two states. Moreover, the black matrix can be simply considered as opaque state. The coding mask is randomly distributed, showing that ∼50% pixels in the LCD are in transparent state and other pixels are in opaque state. However, even for a pixel in transparent state, only 50% light can pass through this LCD pixel, revealing that light efficiency is ∼50%. In addition, monochrome and color CCD cameras (Pike F1100B/C, Allied Vision, Germany) with pixels of 4008 × 2672 and pixel size of 9 µm × 9 µm were used for coded image recording. If the CCD camera is a monochrome one, the NoRDS-CAIC can be monochrome; and if the CCD camera is a color one, the NoRDS-CAIC can be colored. Besides, a router connected to the Raspberry Pi was used to provide the wireless network for coding mask transferring.
Fig. 3. (a) The scheme of the NoRDS-CAIC with three-dimensional shell. (b) Integrated NoRDS-CAIC device with three-dimensional (3-D) printed shell.
In order to integrate the programmable LCD and the CCD camera into a compact NoRDS-CAIC, 3-D printed shell was designed and fabricated as shown in Fig. 3(b). The NoRDS-CAIC only has the size of 19 cm × 15 cm × 16 cm, which is rather compact and can be even used for on-site applications. Both the programmable LCD and the CCD camera can be fixed in the 3-D printed shell, and the NoRDS-CAIC assemble is rather simple, illustrating that the programmable LCD and the CCD camera can be simply assembled into or disassembled from the integrated NoRDS-CAIC. Moreover, an image recorder setting adapter was fabricated, so that in order to switch to a different image recorder, one just needs to use another image recorder setting adapter to assemble its matched image recorder into the NoRDS-CAIC. Moreover, the distance between the programmable LCD and the image recorder can be also adjusted by shifting the image recorder setting adapter, thus guaranteeing that the information of PSFs and coded images can be both recorded. Here in this work, an image recorder setting adapter matching both the monochrome CCD camera (Pike F1100B, Allied Vision, Germany) and the color CCD camera (Pike F1100C, Allied Vision, Germany) was fabricated and the distance between the programmable LCD and the CCD camera target was fixed at 35 mm.
When using the NoRDS-CAIC, an extra computer (laptop or desktop) is also required to control both the programmable LCD and the CCD camera. In the NoRDS-CAIC applications, first, an optimized coding mask is sent to the Raspberry Pi through the wireless network provided by the built-in router, and after the coding mask is displayed on the programmable LCD, the CCD camera captures the corresponding coded image, and transfers it to the computer through the Firewire cable. Next, the above process is cycled until capturing enough coded images (often 30). Finally, the target image can be reconstructed from these captured coded images with the approved NoRDS-CAIC decoding algorithm. In order to use the NoRDS-CAIC automatically, we also designed the NoRDS-CAIC software which is explained in the following section in details.
3.2 Software design of the NoRDS-CAIC
Besides the NoRDS-CAIC hardware construction, we also designed the NoRDS-CAIC software as shown in Fig. 4. This software used Python to develop the main interface, and the core algorithm of image processing was written in MATLAB, and dynamic link library was generated for Python. The software was installed on a desktop with an Intel Core i5 CPU and a 4 GB RAM. The software can automatically detect the PSFs of the coding masks as well as reconstruct the target image from the captured coded images. Moreover, parameters like the exposure time in image recording and the iteration number in target reconstruction can also be adjusted in this software. By connecting the NoRDS-CAIC hardware to a computer installed with the NoRDS-CAIC software, the NoRDS-CAIC can be used for imaging applications.
4. Experimental verification of the NoRDS-CAIC
4.1 Measurements on the point spread functions of the coding masks
After the assembly and the adjustment of the NoRDS-CAIC, it can be used for imaging. But before its applications, the PSFs of the coding masks should be obtained in prior for further target reconstruction. In order to measure these PSFs, a pinhole (Daheng Optics, China) with diameter of 100 µm was first set at a specific distance as 120 mm away from the programmable LCD and was illuminated by a white light source (Daheng Optics, China), thus generating a point source. After setting the point source, open the NoRDS-CAIC as well as its software, all the different coding masks (36 in this work) were sequentially sent to the Raspberry Pi through the wireless network and displayed on the programmable LCD, afterwards, the corresponding coded images were captured using the CCD camera and transferred to the computer through the Firewire cable. Part of the captured coded images in both monochrome and color conditions are listed in Fig. 5. It is worth noting that these captured coded images could be treated as the PSFs corresponding to different coding masks at the specific distances, and these obtained PSFs can only be used to reconstruct the targets still located at this position. In other words, if the targets under detection are located at another distance, PSFs corresponding to that distance still should be measured repeatedly.
4.2 Test on the reconstruction resolution and SNR of the NoRDS-CAIC
The reconstruction resolution and SNR were quantitatively evaluated with a standard USAF 1951 resolution target (Thorlabs, US). Here, the monochrome CCD camera was adopted, and the resolution target was set 120 mm away from the programmable LCD and illuminated by the same white light source (Daheng Optics, China), thus the previously measured PSFs could be used. Here, 36 coded images were captured by varying 36 coding masks, and five coded images are listed in Fig. 6(a). Figures 6(b) to 6(f) list the reconstruction results via single-shot based deconvolution method, Richardson-Lucy method, BM3D aided deconvolution method, multi-exposure based deconvolution method, and NoRDS-CAIC decoding method, respectively. When using the single-shot based deconvolution method, Element 1 in Group 2 (∼125 µm) could be resolved, and the reconstruction result was rather noisy as shown in Fig. 6(b). When using the Richardson-Lucy method, the ringing effect occurred and the resolution was also ∼125 µm since Element 1 in Group 2 could be resolved as shown in Fig. 6(c). Reconstruction result shown in Fig. 6(d) was generated by BM3D aided deconvolution method, where the noise was remarkably reduced, and the resolution reached 111 µm revealed by the resolved Element 2 in Group 2. Reconstruction result shown in Fig. 6(e) was obtained by adding together 36 separately decoded images using single-shot deconvolution method, the noise was remarkably reduced while the resolution remained as 125 µm. However, when using the proposed NoRDS-CAIC decoding algorithm, Element 3 in Group 2 could be resolved from 36 coded images, illustrating a resolution of 99.2 µm shown in Fig. 6(f). Moreover, the reconstruction PSNRs of the reconstructed target images were also evaluated from Figs. 6(b) to 6(f) by simply calculating the PSNRs in the red dotted FoIs. The background of the resolution test chart is chrome plated while some lines and patterns are not, thus light can only pass through these non-chrome plated areas. For those red dotted FoIs without chrome plating, the intensity values in these FoIs should be uniformly distributed theoretically. The calculated MSEs in these FoIs are 0.1105, 0.1391, 0.1032, 0.0974 and 0.0114, corresponding to the PSNRs of 9.57 dB, 8.57 dB, 9.86 dB, 10.11 dB and 19.43 dB, respectively. According to the experiments, it is shown that the spatial resolution and the SNR are improved by 1.3 times and 2 times compared to traditional CAI methods, respectively.
Fig. 6. Experimental results. (a) Representative coded images. Reconstruction results using (b) deconvolution method; (c) Richardson-Lucy method; (d) BM3D aided deconvolution method; (e) multi-exposure method and (f) NoRDS-CAIC decoding method. The inserted images are zoomed for observation. (g) Reconstruction results using NoRDS-CAIC decoding method by using 5, 10, 20, 30 and 36 coded images, respectively. The inserts are zoomed for observation.
Figure 6(g) shows the reconstruction results using the proposed NoRDS-CAIC decoding method from 5, 10, 20, 30 and 36 coded images. According to the reconstruction resolution and SNR data in different conditions, the reconstruction quality was improved with more captured coded images, but when the captured coded image number reached 30, there were hardly significantly improvements in reconstruction quality, all coinciding with the numerical simulations. It is revealed that when using the NoRDS-CAIC, at least 30 coded images are required to reconstruct the target image with the satisfied quality. In NoRDS-CAIC, 36 coding masks are used for recording 36 coded images for target reconstruction.
According to the experimental results using the standard USAF 1951 resolution target, it is demonstrated that the NoRDS-CAIC reached the reconstruction resolution of 99.2 µm and the reconstruction SNR of 19.43 dB, which are improved by 1.3 times and 2 times compared to traditional CAI decoding methods, respectively. Moreover, at least 30 coded images should be captured with varying coding masks to reconstruct the target image with satisfied quality. According to the test on the reconstruction resolution and SNR of the NoRDS-CAIC, it can be well applied in imaging applications, which are discussed in the following section.
4.3 Applications of the NoRDS-CAIC
After the verification on the NoRDS-CAIC, it was finally used for imaging applications. Two samples as a plastic plate and color numbers (written by whiteboard markers) on the glass plate were used, and the color CCD camera was adopted here. Though LCD has strong dispersion on each color, it has slight effect on NoRDS-CAIC since both PSFs and coded images are recorded correspondingly. As in color imaging condition, the target images in three channels (R/G/B) are decoded separately, and the color image is then merged from these decoded RGB images. Figure 7(a) shows representative measured PSF as well as captured coded images, and Figs. 7(b) and 7(c) list the target images reconstructed via different approaches as single-shot based deconvolution method, Richardson-Lucy method, BM3D aided deconvolution method, multi-exposure based deconvolution method (from 36 coded images) and NoRDS-CAIC decoding method (from 36 coded images). The reconstruction results seemed grainy mainly due to the roughness of the plastic plate and the nonuniformity of the ink on the glass plate. Similar to the numerical simulation results, the proposed NoRDS-CAIC obtained the highest reconstruction quality with both enhanced resolution and SNR, supporting the effectiveness of the NoRDS-CAIC in imaging applications.
Fig. 7. (a) Representative coded images with corresponding coding mask and measured PSF. (b) and (c) are reconstruction results using deconvolution method, richardson-Lucy method, BM3D aided deconvolution method, multi-exposure method and NoRDS-CAIC decoding method, respectively. (d) and (e) are reconstruction results with NoRDS-CAIC from 5, 10, 20, 30 and 36 coded images, respectively.
Finally, we also checked the relation between the reconstruction quality and the coded image number in practical sample cases. Reconstructed target images from 5, 10, 20, 30 and 36 coded images are shown in Figs. 7(d) and 7(e), revealing that with more coded images, the reconstruction quality improved, but when the coded image number reached 30, the reconstruction quality became stable. The results coincide with the previous simulations and experiments, showing that the NoRDS-CAIC requires at least 30 coded images for imaging applications.
5. Discussion
5.1 Total time for NoRDS-CAIC
According to the frame rate of the programmable LCD and the image rate of CCD, ∼4 s was needed for capturing 36 coded images (here, the time consumed on PSFs measurement is not considered, since the process can be measured in prior). And in data processing, the decoding time was ∼6 s with a moderate-configuration computer (Intel Core i5-6500 CPU at 3.20 GHz and 4 GB RAM). Thus, a total of ∼10 s was required for NoRDS-CAIC and is nearly 4 times faster than our previously proposed method [41]. However, the total time required for NoRDS-CAIC can be further reduced by accelerating both image recording and data processing. In image recording, if the frame rates of the programmable LCD and CCD camera become higher, the time in image recording can be reduced. Additionally, in data processing, the time consuming can be decreased with accelerated techniques such as GPU-aided data processing [43]. However, by increasing the frame rates, less light intensity can be collected thus inevitably reducing SNR. Moreover, GPU-aided data processing requires expensive and complicated hardware.
5.2 Influence of dispersion, distance and temperature for NoRDS-CAIC
Firstly, in NoRDS-CAIC, though the programmable LCD has dispersion effect on different wavelengths, the decoded images reconstructed by NoRDS-CAIC still show good dispersion suppression as shown in Fig. 7, which is because both PSFs and coded images were recorded correspondingly. In order to verify the dispersion suppression capability of the NoRDS-CAIC, experiments using the standard USAF 1951 resolution target were implemented using the illuminations (LED sources, Daheng Optics, China) with the central wavelengths of 451 nm, 528 nm and 620 nm, respectively, combining with a colored CCD camera. Figure 8(a) shows the reconstruction results as well as their merged result, which reveal that target images in different illumination wavelengths can be all reconstructed in high quality. The results in Fig. 7 and Fig. 8(a) support that the NoRDS-CAIC has good dispersion suppression capability to resist the effect of LCD dispersion.
Fig. 8. (a) Reconstruction results with red, green, blue light illuminations, respectively, and their merged result. (b) Reconstruction results with different distances between the programmable LCD and the CCD camera. (c) Reconstruction results at different temperatures.
Secondly, the NoRDS-CAIC can also work well in conditions with different distances between the programmable LCD and the CCD camera. In order to prove it, experiments using the standard USAF 1951 resolution target were implemented with the distances of 35 mm and 50 mm, respectively. Figure 8(b) shows the reconstruction results, and their reconstruction qualities were rather similar, proving that NoRDS-CAIC can still work well even changing its inner structure such as the distance between the programmable LCD and the CCD camera. However, when the programmable LCD is set further from the CCD camera, partial information of coded images may loss with limited target size of CCD camera, therefore, the programmable LCD should not be located rather far from the CCD camera.
Thirdly, considering that the programmable LCD is temperature dependent, we applied the NoRDS-CAIC for lensless imaging at different temperatures as 10 °C, 20 °C and 30 °C maintained using air conditioner, and Fig. 8(c) shows the results all revealed with high reconstruction qualities. It is worth noting that both point spread function and coded image were recorded at the same temperature. According to the results, it shows that the NoRDS-CAIC could work well in room temperature as from 10 °C to 30 °C.
5.3 Some limitations for NoRDS-CAIC
In this work, the programmable LCD shows the capacity of varying coding masks dynamically. Besides, SLM [22–26] and DMD [21] can also vary the coding masks rapidly and precisely; however, they are much more expensive than the programmable LCD. Nevertheless, it should be mentioned that the programmable LCD still has some limitations. The electronic elements on LCD pixels will block light and lead to low light efficiency (∼50%), thus the light loss will result in a decreased SNR in return. Besides, when the coding mask is displayed on the programmable LCD, each pixel on LCD does not reveal the complete states of “transparent” or “opaque”, thus not only reducing the contrast, but also introducing residual background to the captured coded images. But according to the experimental results, the proposed NoRDS-CAIC could provide reconstructed target images with high quality, therefore, the proposed NoRDS-CAIC is still a potential tool in imaging applications.
Since the light efficiency can be improved by a phase coded mask instead of an amplitude mask as presented in DiffuserCam [28,29], a well-designed coding mask which introduces phase modulation on the programmable LCD may be potentially applied to improve the light efficiency of NoRDS-CAIC.
Additionally, commercial CCD camera and adjustable LCD are used in our proposed NoRDS-CAIC rather than CCD chip and fixed coding mask used in FlatCam [17–19], therefore, NoRDS-CAIC is larger and more expensive than FlatCam. However, the high-quality commercial CCD camera and the variable coding masks can achieve better reconstruction quality.
In this work, we only focused on the 2-D imaging, the samples used were considered as 2-D objects, thus no depth information was considered. While 3-D CAI techniques have been also proposed such as DiffuserCam [28,29]. The proposed method can be future expanded to 3-D imaging, but in this case, 3-D PSF distribution should be measured, and the suggested NoRDS-CAIC decoding algorithm should also be updated.
5.4 Comparisons to other CAI methods
The programmable LCD can obtain amplitude modulation and has been used for lensless imaging [16], while SLM has the capability in both amplitude and phase modulation, therefore, besides the applications in classical CAI, SLM has also been extended in new type lensless imaging as coded aperture correlation holography [30–37]. However, our proposed NoRDS-CAIC is totally different from coded aperture correlation holography devices. (1) According to the principle of our designed NoRDS-CAIC, it reconstructs the target image through deconvolution using iterative Wiener filter, therefore NoRDS-CAIC is completely a non-interferometric device. While coded aperture correlation holography relies on interference between two mutually coherent beams split from each object point, and the generated hologram contains the information about the 3-D location of the object point. (2) In NoRDS-CAIC, the used LCD only acts as the transmission mask with only transparent and opaque states; while in coded aperture correlation holography system, a SLM is required to generate a quasi-random phase function, and it is worth noting that phase-only modulated based SLM is much more expensive than amplitude-only modulated based LCD. But it is still worth noting that coded aperture correlation holography provides another perspective for CAI.
In addition, most of the classical CAI works only focus on intensity imaging, while losing phase information which is another part of the wavefront especially in coherent imaging conditions. Our proposed NoRDS-CAIC also ignores the phase shift introduced by LCD, however, the phase shift is not important since only the incoherent PSF is used for image decoding. Though Veeraraghavan group [22], Heidrich group [23] and Horisaki group [24] expand CAI into wavefront sensing, there are still less work on phase imaging, especially only relying on amplitude modulated coding masks. It is the future work that the NoRDS-CAIC can be updated to wavefront imaging and sensing device.
6. Conclusion
In summary, we have demonstrated the NoRDS-CAIC as a lensless imaging device. The NoRDS-CAIC integrated with a programmable LCD and an image recorder is cost-effective and compact only with the size of 19 cm × 15 cm × 16 cm, and the target sample can be reconstructed using our proposed NoRDS-CAIC decoding algorithm proved with both enhanced resolution and SNR. The NoRDS-CAIC can be controlled using our designed software which can automatically display the coding masks, capture the coded images and reconstruct the target image. In addition, the programmable LCD can load and display the coding masks as binary pinhole arrays precisely and rapidly, significantly accelerating the imaging recording speed. Verified by visible light experiments, the reconstruction resolution and SNR can reach 99.2 µm and 19.43 dB, which are improved by ∼1.3 times and ∼2 times compared to the traditional CAI decoding method. Considering its advantages as compact system, cost-effective configuration, fast recording speed and high reconstruction quality, it is shown that the designed NoRDS-CAIC can be potentially used for lensless imaging in practical applications.
Appendix
Simulation Matlab source code
The simulation Matlab code consists of the following 3 steps: 1) load object image; 2) set parameters; 3) image coding and decoding. The detailed source code is listed as follows.
$\color{green}{\%\%\textrm{ }\textbf{step 1: load object image}}$
image = double(rgb2gray(imread(‘$\color{purple}{\textrm{siemensstar.jpg}}$’))); $\color{green}{\%\textrm{ }\textrm{load image}}$
image = image./max(max(image)); $\color{green}{\%\textrm{ }\textrm{image intensity normalization}}$
$\color{green}{\%\%\textrm{ }\textbf{step 2: set parameters}}$
lambda = 0.6328; $\color{green}{\%\textrm{ }\textrm{wavelength }({\textrm{micrometer}} )}$
k = 2*pi/lambda; $\color{green}{\%\textrm{ }\textrm{wave vector}}$
pixelsize = 7.4; $\color{green}{\%\textrm{ }\textrm{Effective pixel size of imaging system}}$
di = 150e3; $\color{green}{\%\textrm{ }\textrm{distance:object to aperture}}$
do = 100e3; $\color{green}{\%\textrm{ }\textrm{distance: aperture to CCD}}$
p = size(image,1); $\color{green}{\%\textrm{ }\textrm{pixel number}}$
ax=-p/2:p/2-1;
[x,y] = meshgrid(ax,ax); $\color{green}{\%\textrm{ }\textrm{meshgrid}}$
$\color{green}{\%\%\textrm{ }\textbf{step 3: image coding and decoding}}$
f_image_guess = ones(p); $\color{green}{\%\textrm{ }\textrm{initial object guess}}$
cnumber = 36; $\color{green}{\%\textrm{ }\textrm{set the number of coding mask}}$
for ii = 1:cnumber
$\color{green}{\%\textrm{ }\textbf{step 3 - 1:set random distributed coding mask}}$
p0 = p/2;
N = 2;
mask = round(rand(p0/N,p0/N));
mask = imresize(mask,N,'nearest’); $\color{green}{\%\textrm{ }\textrm{set random distributed binary mask}}$
aperture = zeros(p,p);
aperture(p/2-p0/2 + 1 + mm0:p/2 + p0/2 + mm0,p/2-p0/2 + 1 + nn0:p/2 + p0/2 + nn0)=mask;
$\color{green}{\%\textrm{setcoding mask}}$
$\color{green}{\%\textbf{step 3 - 2:generate coded images and corresponding PSFs} \& \textrm{OTFs}}$
[h, H_CTF, H_OTF, I_coherent, I_incoherent] = incoherent_image_generate(lambda, image, di, do, x, y, pixelsize, aperture);
$\color{green}{\%\textbf{step 3 - 3: image decoding with NoRDS - CAIC decoding algorithm}}$
f_image0 = f_image_guess;
f_img = fftshift(fft2(fftshift(I_incoherent)));
change = f_img-f_image0.*H_OTF;
c_H = abs(H_OTF).^2;
eps = 10^-5; $\color{green}{\%\textrm{ }\textrm{regularization parameter}}$
f_image = f_image0 + conj(H_OTF)./(abs(H_OTF).^2 + eps*max(max(c_H))).
*change;$\color{green}{\%\textrm{ }\textrm{update in fourier domain}}$
re_image = abs(ifftshift(ifft2(f_image)));
$\color{green}{\%\textrm{decoded image with NoRDS - CAIC decoding algorithm}}$
f_image_guess = f_image; $\color{green}{\%\textrm{ }\textrm{used for next iteration}}$
$\color{green}{\%\textrm{ }\textbf{step 3 - 4: - image decoding with deconvolution method }({\textrm{for comparison}} )}$
dir_decode = incoherent_decoding(I_incoherent, H_OTF, eps);
$\color{green}{\%\textrm{decoded image with deconvolution method}}$
figure(1);
subplot(221);imagesc(h);axis image;title(‘$\color{purple}{\textrm{PSF image}}$’);
subplot(222);imagesc(I_incoherent);axis image;title(‘$\color{purple}{\textrm{Coded image}}$’);
subplot(223);imagesc(dir_decode);axis image;
title(‘$\color{purple}{\textrm{Decoded image with deconvolution method}}$’);
subplot(224);imagesc(re_image);axis image;
title(‘$\color{purple}{\textrm{Decoded image with NoRDS - CAIC decoding algorithm}}$’);
end
$\color{green}{\%\%\textrm{ }\textbf{function 1 - incoherent}\_\textrm{image}\_\textrm{generate}}$
$\color{pink}{\textrm{function}}$ [ h, H_CTF, H_OTF, I_coherent, I_incoherent ] =
incoherent_image_generate(lambda, obj, di, do, x, y, pixelsize, aperture)
$\color{green}{\%\textrm{ generate PSF}}$
k = 2*pi/lambda;
diff1 = 1/di*exp(1j*k*di).*exp(1j*k*((x*pixelsize).^2+(y*pixelsize).^2)/di/2);
diff2 = diff1.*aperture;
diff3 = 1./(1j*lambda*do)*exp(1j*k*do).
*exp(1j*k*((x*pixelsize).^2+(y*pixelsize).^2)/do/2).
*fftshift(fft2(fftshift(diff2.*exp(1j*k*((x*pixelsize).^2+(y*pixelsize).^2)/do/2))));
h0 = diff3;
h = diff3.*conj(diff3); $\color{green}{\%\textrm{ }\textrm{system PSF}}$
H1 = fftshift(fft2(fftshift(h0))); $\color{green}{\%\textrm{ }\textrm{CTF}}$
H2 = fftshift(fft2(fftshift(h)));
H2 = abs(H2)./max(max(abs(H2))); $\color{green}{\%\textrm{ }\textrm{OTF}}$
$\color{green}{\%\textrm{ }\textrm{coherent and incoherent imaging}}$
f_o = fftshift(fft2(fftshift(obj)));$\color{green}{\%\textrm{ }\textrm{spectrum of object}}$
$\color{green}{\%\textrm{ }\textrm{generate image under coherent illumination}}$
f1 = f_o.*H1;
u1 = ifftshift(ifft2(ifftshift(f1)));
I1 = abs(u1).^2; $\color{green}{\%\textrm{ }\textrm{coherent}}$
$\color{green}{\%\textrm{ }\textrm{generate image under incoherent illumination}}$
f2 = f_o.*H2;
u2 = ifftshift(ifft2(ifftshift(f2)));
I2 = abs(u2); $\color{green}{\%\textrm{ }\textrm{incoherent}}$
H_CTF = H1;
H_OTF = H2;
I_coherent = I1;
I_incoherent = I2;
$\color{pink}{\textrm{end}}$
$\color{green}{\%\%\textrm{ }\textbf{function 2 - incoherent}\_\textrm{decoding}}$
$\color{pink}{\textrm{function}}$ [ re_I0, f_re_I0 ] = incoherent_decoding(I_incoherent,H_OTF,eps)
$\color{green}{\%\textrm{ }\textrm{image decoding with deconvolution method}}$
I_code = I_incoherent;
f_I = fftshift(fft2(fftshift(I_code)));
HHH = H_OTF; %% OTF
c_H = abs(HHH).^2;
f_re_I0 = conj(HHH)./(abs(HHH).^2 + eps*max(max(c_H))).*f_I;
re_I0 = ifftshift(ifft2(ifftshift(f_re_I0)));
re_I0 = abs(re_I0);
$\color{pink}{\textrm{end}}$
Funding
National Natural Science Foundation of China (NSFC) (61705092, 11947094, U1730132, 11804263); Natural Science Foundation of Jiangsu Province of China (BK20180598, BK20170194); Fundamental Research Funds for the Central Universities (JUSRP11839, JUSRP51721B).
Disclosures
The authors declare no conflicts of interest.
References
1. R. H. Dicke, “Scatter-hole cameras for x-rays and gamma rays,” Astrophys. J. 153, L101–L106 (1968). [CrossRef]
2. J. G. Ables, “Fourier transform photography: a new method for x-ray astronomy,” Publ. Astron. Soc. Aust. 1(4), 172–173 (1968). [CrossRef]
3. H. H. Barrett, D. T. Wilson, and G. D. Demeester, “The use of half-tone screens in Fresnel zone plate imaging of incoherent sources,” Opt. Commun. 5(5), 398–401 (1972). [CrossRef]
4. H. H. Barrett and F. A. Horrigan, “Fresnel zone plate image of gamma rays; theory,” Appl. Opt. 12(11), 2686–2702 (1973). [CrossRef]
5. R. Accorsi and R. C. Lanza, “Near-field artifact reduction in coded aperture imaging,” Appl. Opt. 40(26), 4697–4705 (2001). [CrossRef]
6. R. Accorsi, F. Gasparini, and R. C. Lanza, “Optimal coded aperture patterns for improved SNR in nuclear medicine imaging,” Nucl. Instrum. Methods Phys. Res., Sect. A 474(3), 273–284 (2001). [CrossRef]
7. R. Accorsi, F. Gasparini, and R. C. Lanza, “A coded aperture for high-resolution nuclear medicine planar imaging with a conventional anger camera: experimental results,” IEEE Trans. Nucl. Sci. 48(6), 2411–2417 (2001). [CrossRef]
8. M. J. Cieślak, K. A. A. Gamage, and R. Glover, “Coded-aperture imaging systems: Past, present and future development–A review,” Radiat. Meas. 92, 59–71 (2016). [CrossRef]
9. R. K. Mortimer, H. O. Anger, and C. A. Tobias, “The gamma-ray pinhole camera with image amplifier,” Radiation Lab, Univ. of Calif, Berkeley (1954).
10. M. Feroci, E. Costa, P. Soffitta, E. Del Monte, G. Di Persio, I. Donnarumma, Y. Evangelista, M. Frutti, I. Lapshov, M. Mastropietro, E. Morelli, L. Pacciani, G. Porrovecchio, M. Rapisarda, A. Rubini, M. Tavani, and A. Argan, “SuperAGILE: The hard X-ray imager for the AGILE space mission,” Nucl. Instrum. Methods Phys. Res., Sect. A 581(3), 728–754 (2007). [CrossRef]
11. O. Gal, B. Dessus, F. Jean, F. Laine, and C. Leveque, “Operation of the CARTOGAM portable gamma camera in a photon-counting mode,” IEEE Trans. Nucl. Sci. 48(4), 1198–1204 (2001). [CrossRef]
12. A. Schwarz, A. Shemer, and Z. Zalevsky, “Light intensity and SNR improvement for high-resolution optical imaging via time multiplexed pinhole arrays,” Appl. Opt. 53(20), 4483–4492 (2014). [CrossRef]
13. E. E. Fenimore and T. M. Cannon, “Coded aperture imaging with uniformly redundant arrays,” Appl. Opt. 17(3), 337–347 (1978). [CrossRef]
14. E. E. Fenimore and T. M. Cannon, “Uniformly redundant arrays: digital reconstruction methods,” Appl. Opt. 20(10), 1858–1864 (1981). [CrossRef]
15. E. E. Fenimore, “Coded aperture imaging: the modulation transfer function for uniformly redundant arrays,” Appl. Opt. 19(14), 2465–2471 (1980). [CrossRef]
16. A. Zomet and S. K. Nayar, “Lensless imaging with a controllable aperture,” In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06) (Vol. 1, pp. 339–346). IEEE. (2006, June).
17. J. Tan, L. Niu, J. K. Adams, V. Boominathan, J. T. Robinson, R. G. Baraniuk, and A. Veeraraghavan, “Face Detection and Verification Using Lensless Cameras,” IEEE T. Comp. Imag. 5(2), 180–194 (2019). [CrossRef]
18. M. S. Asif, A. Ayremlou, A. Sankaranarayanan, A. Veeraraghavan, and R. G. Baraniuk, “Flatcam: Thin, lensless cameras using coded aperture and computation,” IEEE T. Comp. Imag. 3(3), 384–397 (2017). [CrossRef]
19. J. K. Adams, V. Boominathan, B. W. Avants, D. G. Vercosa, F. Ye, R. G. Baraniuk, and A. Veeraraghavan, “Single-frame 3D fluorescence microscopy with ultraminiature lensless FlatScope,” Sci. Adv. 3(12), e1701548 (2017). [CrossRef]
20. Z. Wang and G. William, “Coded pinhole lens imaging,” OSA Continuum 1(1), 64–77 (2018). [CrossRef]
21. X. Cao, T. Yue, X. Lin, S. Lin, X. Yuan, Q. Dai, and D. J. Brady, “Computational snapshot multispectral cameras: Toward dynamic capture of the spectral world,” IEEE Signal Proc. Mag. 33(5), 95–108 (2016). [CrossRef]
22. Y. Wu, M. K. Sharma, and A. Veeraraghavan, “WISH: wavefront imaging sensor with high resolution,” Light: Sci. Appl. 8(1), 44 (2019). [CrossRef]
23. C. Wang, X. Dun, Q. Fu, and W. Heidrich, “Ultra-high resolution coded wavefront sensor,” Opt. Express 25(12), 13736–13746 (2017). [CrossRef]
24. R. Horisaki, R. Egami, and J. Tanida, “Experimental demonstration of single-shot phase imaging with a coded aperture,” Opt. Express 23(22), 28691–28697 (2015). [CrossRef]
25. W. Chi and N. George, “Phase-coded aperture for optical imaging,” Opt. Commun. 282(11), 2110–2117 (2009). [CrossRef]
26. W. Chi and N. George, “Optical imaging with phase-coded aperture,” Opt. Express 19(5), 4294–4300 (2011). [CrossRef]
27. H. Y. Liu, J. Zhong, and L. Waller, “Multiplexed phase-space imaging for 3d fluorescence microscopy,” Opt. Express 25(13), 14986–14995 (2017). [CrossRef]
28. G. Kuo, N. Antipa, R. Ng, and L. Waller, “Diffusercam: diffuser-based lensless cameras,” In Computational Optical Sensing and Imaging (pp. CTu3B-2). Optical Society of America. (2017, June).
29. N. Antipa, G. Kuo, R. Heckel, B. Mildenhall, E. Bostan, R. Ng, and L. Waller, “DiffuserCam: lensless single-exposure 3D imaging,” Optica 5(1), 1–9 (2018). [CrossRef]
30. A. Vijayakumar and J. Rosen, “Interferenceless coded aperture correlation holography – a new technique for recording incoherent digital holograms without two-wave interference,” Opt. Express 25(12), 13883–13896 (2017). [CrossRef]
31. M. Kumar, A. Vijayakumar, and J. Rosen, “Incoherent digital holograms acquired by interferenceless coded aperture correlation holography system without refractive lenses,” Sci. Rep. 7(1), 11555 (2017). [CrossRef]
32. M. R. Rai, A. Vijayakumar, and J. Rosen, “Single camera shot interferenceless coded aperture correlation holography,” Opt. Lett. 42(19), 3992–3995 (2017). [CrossRef]
33. M. R. Rai, A. Vijayakumar, and J. Rosen, “Non-linear adaptive three-dimensional Imaging with interferenceless coded aperture correlation holography (I-COACH),” Opt. Express 26(14), 18143–18154 (2018). [CrossRef]
34. M. R. Rai, A. Vijayakumar, Y. Ogura, and J. Rosen, “Resolution enhancement in nonlinear interferenceless COACH with point response of subdiffraction limit patterns,” Opt. Express 27(2), 391–403 (2019). [CrossRef]
35. J. Rosen, V. Anand, M. R. Rai, S. Mukherjee, and A. Bulbul, “Review of 3D imaging by coded aperture correlation holography (COACH),” Appl. Sci. 9(3), 605 (2019). [CrossRef]
36. M. R. Rai, A. Vijayakumar, and J. Rosen, “Superresolution beyond the diffraction limit using phase spatial light modulator between incoherently illuminated objects and the entrance of an incoherent imaging system,” Opt. Lett. 44(7), 1572–1575 (2019). [CrossRef]
37. M. R. Rai and J. Rosen, “Noise suppression by controlling the sparsity of the point spread function in interferenceless coded aperture correlation holography (I-COACH),” Opt. Express 27(17), 24311–24323 (2019). [CrossRef]
38. Z. Zhao, Y. Ding, J. Dong, Y. Hao, S. Wu, L. Cao, and Y. Pu, “Richardson–Lucy method for decoding x-ray ring code image,” Plasma Phys. Controlled Fusion 49(8), 1145–1150 (2007). [CrossRef]
39. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3-D transform-domain collaborative filtering,” IEEE T. Image. Process. 16(8), 2080–2095 (2007). [CrossRef]
40. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising with block-matching and 3D filtering,” Proc. SPIE 6064, 606414 (2006). [CrossRef]
41. Z. Jiang, Y. Kong, W. Qian, S. Wang, and C. Liu, “Resolution and signal-to-noise ratio enhancement for synthetic coded aperture imaging via varying pinhole array,” Appl. Opt. 58(22), 6157–6164 (2019). [CrossRef]
42. J. W. Goodman, Introduction to Fourier Optics (Roberts and Company, 2005).
43. Z. Jiang, Y. Kong, F. Liu, C. Liu, and S. Wang, “Graphics processing unit (GPU) aided wavefront-based autofocusing in microscopy,” AIP Adv. 8(10), 105328 (2018). [CrossRef]
