"Pixel-by-pixel absolute phase retrieval assisted by an additional three-dimensional scanner," Appl. Opt., (2019)

Y. An and S. Zhang, “Pixel-by-pixel absolute phase retrieval assisted by an additional three-dimensional scanner”, Appl. Opt., 58(8), 2033-2041, 2019, doi:10.1364/AO.58.002033

Abstract

This paper presents a novel absolute phase unwrapping method assisted by a low-cost three-dimensional (3D) scanner. The proposed absolute phase unwrapping method leverages a low-cost 3D scanner to capture rough 3D data of the scene, and transforms the rough 3D data to the world coordinate system to generate an artificial reference phase map Φ_. By referring to Φ_, we can do absolute phase unwrapping directly without projecting any additional patterns, such that the digital fringe projection (DFP) system can achieve higher measurement speed. We develop a multi-resolution system consisting of a DFP system and Kinect V2 to validate our method. Experiments demonstrate that our method works for a large depth range, and the speed of the low-cost 3D scanner is not necessarily the maximum speed of our proposed method. Assisted by Kinect V2 whose maximum speed is only 30Hz, our DFP system achieves 53Hz with a resolution 1600x1000 pixels when we measure dynamic objects that are moving in a large depth range of 400mm.

"Motion-induced error compensation for phase shifting profilometry," Opt. Express (2018)

[114] Z. Liu, P. Zibley and S. Zhang, "Motion-induced error compensation for phase shifting profilometry," Opt. Express, 26(10), 12632-12637 (2018); doi:10.1364/OE.26.012632

Abstract

This paper proposes a novel method to substantially reduce motion-introduced phase error in phase-shifting profilometry. We first estimate the motion of an object from the difference between two subsequent 3D frames. After that, by leveraging the projector’s pinhole model, we can determine the motion-induced phase shift error from the estimated motion. A generic phase-shifting algorithm considering phase shift error is then utilized to compute the phase. Experiments demonstrated that proposed algorithm effectively improved the measurement quality by compensating for the phase shift error introduced by rigid and nonrigid motion for a standard single-projector and single-camera digital fringe projection system.

"Three-dimensional shape measurement using a structured light system with dual projectors", Appl. Opt., (2018)

[113] C. Jiang, B. Lim and S. Zhang, "Three-dimensional shape measurement using a structured light system with dual projectors," Appl. Opt.,  57(14), 3983-3990(2018); doi:10.1364/AO.57.003983

This paper introduces a structured light system with two projectors and one camera for three-dimensional (3D) shape measurement to alleviate problems created by a single projector such as the shadow problem. In particular, we developed (1) a system calibration framework that can accurately calibrate each such camera-projector system; (2) a residual error correction method based on the system error function; and (3) a data fusion method utilizing the angle between the projection direction and surface normal. Experimental results demonstrate that the proposed dual-projector structured light system improves the measurement accuracy besides extending the measurement range of a single projector system.

"High-speed and high-accuracy 3D surface measurement using a mechanical projector," Opt. Express (2018)

J. -S. Hyun, George T. -C. Chiu and S. Zhang, "High-speed and high-accuracy 3D surface measurement using a mechanical projector," Opt. Express, 26(2), 1474-1487 (2018); doi:10.1364/OE.26.001474

Abstract

This paper presents a method to achieve high-speed and high-accuracy 3D surface measurement using a custom-designed mechanical projector and two high-speed cameras. We developed a computational framework that can achieve absolute shape measurement in sub-pixel accuracy through: 1) capturing precisely phase-shifted fringe patterns by synchronizing the cameras with the projector; 2) generating a rough disparity map between two cameras by employing a standard stereo-vision method using texture images with encoded statistical patterns; and 3) utilizing the wrapped phase as a constraint to refine the disparity map. The projector can project binary patterns at a speed of up to 10,000 Hz, and the camera can capture the required number of phase-shifted fringe patterns with 1/10,000 second, and thus 3D shape measurement can be realized as high as 10,000 Hz regardless the number of phase-shifted fringe patterns required for one 3D reconstruction. Experimental results demonstrated the success of our proposed method.

"Three-dimensional range geometry compression via phase encoding," Appl. Opt. (2017)

[106] T. Bell, B. Vlahov, J.P. Allebach, and S. Zhang, "Three-dimensional range geometry compression via phase encoding," Appl. Opt., 56(33), 9285-9292, (2017); doi: 10.1364/AO.56.009285

Abstract

One of the state-of-the-art methods for three-dimensional (3D) range geometry compression is to encode 3D data within a regular 24-bit 2D color image. However, most existing methods use all three color channels to solely encode 3D data, leaving no room to store other information (e.g., texture) within the same image. This paper presents a novel method which utilizes geometric constraints, inherent to the structured light 3D scanning device, to reduce the amount of data which need be stored within the output image. The proposed method thus only requires two color channels to represent 3D data, leaving one channel free to store additional information (such as a texture image). Experimental results verify the overall robustness of the proposed method. For example, a compression ratio of 3038:1 can be achieved, versus the STL format, with a root-mean-square (RMS) error of 0.47% if the output image is compressed with JPEG 80%.

Technical Paper

“Absolute three-dimensional shape measurement with two-frequency square binary patterns,” Appl. Opt., (2017)

C. Jiang and S. Zhang, “Absolute three-dimensional shape measurement with two-frequency square binary patterns,” Appl. Opt., 56(31), 8710-8718 (2017); doi:10.1364/AO.56.008710

Abstract

This paper presents a novel method to achieve absolute three-dimensional (3D) shape measurement solely using square binary patterns. This method uses six patterns: three low-frequency phase-shifted patterns and three phase-shifted high-frequency patterns. The phase obtained from low-frequency phase temporally unwraps the phase obtained from high-frequency patterns. The projector is defocused such that the high-frequency patterns produce high-quality phase, but the phase retrieved from low-frequency patterns has large harmonic error that fails two-frequency temporal phase unwrapping process. In this paper, we develop a computational framework to address the challenge. The proposed computational framework includes four major approaches to alleviate the harmonic error problem: i) use more than one period of low-frequency patterns enabled by geometric constraint-based phase unwrapping method; ii) artificially apply a large Gaussian filter to low frequency patterns before phase computation; iii) create an error lookup table (LUT) to compensate for harmonic error; and iv) develop a boundary error correction method to alleviate problems associated with filtering. Both simulation and experimental results demonstrated the success of the proposed method.

“Absolute phase unwrapping for dual-camera system without embedding statistical features,” Opt. Eng., (2017)

C. Jiang and S. Zhang, “Absolute phase unwrapping for dual-camera system without embedding statistical features,” Opt. Eng. 56(9), 094114 (2017), doi: 10.1117/1.OE.56.9.094114.

Abstract

This paper proposes an absolute phase unwrapping method for 3D measurement that uses two cameras and one projector. On the left camera image, each pixel has one wrapped phase value which corresponds to multiple projector candidates with different absolute phase values. We use geometric relationship of the system to map projector candidates into right camera candidates. By applying a series of candidate rejection criteria, a unique correspondence pair between two camera images can be determined. Then the absolute phase is obtained by tracing the correspondence point back to projector space. Experimental results demonstrate that the proposed absolute phase unwrapping algorithm can successfully work on both complex geometry and multiple isolated objects measurement.

"High-speed high-accuracy three-dimensional shape measurement using digital binary defocusing method versus sinusoidal method," Opt. Eng., (2017)

[99] J. -S. Hyun, B. Li, and S. Zhang, "High-speed high-accuracy three-dimensional shape measurement using digital binary defocusing method versus sinusoidal method," Opt. Eng. 56(7), 074102 (2017).

Abstract

This paper presents our research findings on high-speed high-accuracy 3D shape measurement using digital light processing (DLP) technologies. In particular, we compare two different sinusoidal fringe generation techniques using the DLP projection devices: direct projection of 8-bit computer generated sinusoidal patterns (a.k.a., the sinusoidal method), and the creation of sinusoidal patterns by defocusing binary patterns (a.k.a., the binary defocusing method). This paper mainly examines their performance on high-accuracy measurement applications under precisely controlled settings. Two different projection systems were tested in this study: the commercially available inexpensive projector, and the DLP development kit. Experimental results demonstrated that the binary defocusing method always outperforms the sinusoidal method if a sufficient number of phase-shifted fringe patterns can be used.

"Three-dimensional absolute shape measurement by combining binary statistical pattern matching with phase-shifting methods," Appl. Opt., (2017)

Y. An and S. Zhang, "Three-dimensional absolute shape measurement by combining binary statistical pattern matching with phase-shifting methods," Appl. Opt., (2017); (accepted)

Abstract

This paper presents a novel method that leverages the stereo geometric relationship between projector and camera for absolute phase unwrapping on a standard one-projector and one-camera structured light system. Specifically, we use only one additional binary random image and the epipolar geometric constraint to generate a coarse correspondence map between projector and camera images. The coarse correspondence map is further refined by using the wrapped phase as a constraint. We then use the refined correspondence map to determine a fringe order for absolute phase unwrapping. Experimental results demonstrated the success of our proposed method.

"Pixel-by-pixel absolute three-dimensional shape measurement with modified Fourier transform profilometry" Appl. Opt., (2017)

[95] H. Yun, B. Li, and S. Zhang, "Pixel-by-pixel absolute three-dimensional shape measurement with modified Fourier transform profilometry", Appl. Opt., 56(5), 1472-1480, (2017); doi: 10.1364/AO.56.001472

Abstract

Single-pattern Fourier transform profilometry (FTP) method and double-pattern modified FTP method have great value on high-speed three-dimensional (3D) shape measurement, yet it is difficult to retrieve absolute phase pixel by pixel. This paper presents a method that can recover absolute phase pixel by pixel for the modified FTP method. The proposed method uses two images with different frequencies, and the recovered low frequency phase is used to temporally unwrap the high-frequency phase pixel by pixel. This paper also presents the computational framework to reduce noise impact for robust phase unwrapping. Experiments demonstrate the success of the proposed absolute phase recovery method using only two fringe patterns.

"Pixel-by-pixel absolute phase retrieval using three phase-shifted fringe patterns without markers," Opt. Laser Eng., (2017)

C. Jiang,  B. Li, S. Zhang, "Pixel-by-pixel absolute phase retrieval using three phase-shifted fringe patterns without markers," Opt. Laser Eng., 91, 232-241 (2017);  doi:10.1016/j.optlaseng.2016.12.002

This paper presents a method that can recover absolute phase pixel by pixel without embedding markers on three phase-shifted fringe patterns, acquiring additional images, or introducing additional hardware component(s). The proposed
three-dimensional (3D) absolute shape measurement technique includes the following major steps: 1) segment the measured object into different regions using rough priori knowledge of surface geometry; 2) artificially create phase maps at different z planes using geometric constraints of structured light system; 3) unwrap the phase pixel by pixel for each region by properly referring to the artificially created phase map; and 4) merge unwrapped phases from all regions into a complete absolute phase map for 3D reconstruction. We demonstrate that conventional three-step phase-shifted fringe patterns can be used to create absolute phase map pixel by pixel even for large depth range objects. We have successfully implemented our proposed computational framework to achieve absolute 3D shape measurement at 40 Hz.

"Evaluation of pixel-wise geometric constraints based phase unwrapping method for low signal-to-noise-ratio (SNR) phase," Advanced Optical Technologies, (2016)

[91] Y. An, Z. Liu and S. Zhang, "Evaluation of pixel-wise geometric constraints based phase unwrapping method for low signal-to-noise-ratio (SNR) phase," Advanced Optical Technologies, 5(5-6), 423–432, (2016); doi: 10.1515/aot-2016-0048

This paper evaluates the robustness of our recently proposed geometric constraints based phase unwrapping method to unwrap low signal-to-noise ratio (SNR) phase.  Instead of capturing additional images for absolute phase unwrapping, the new phase unwrapping algorithm uses geometric constraints of the digital fringe projection (DFP) system to create a virtual reference phase map to unwrap the phase pixel by pixel. Both simulation and experimental results demonstrate that this new phase unwrapping method can even successfully unwrap low SNR phase maps that brings difficulties for conventional multi-frequency phase unwrapping methods.

"Superfast 3D absolute shape measurement using five binary patterns," Opt. Laser Eng., (2017)

[93] J. -S. Hyun and S. Zhang, "Superfast 3D absolute shape measurement using five binary patterns," Opt. Laser Eng., 90, 217-224, 2017; 10.1016/j.optlaseng.2016.10.017

Abstract

This paper presents a method that recovers high-quality 3D absolute coordinates point by point with only five binary patterns. Specifically, three dense binary dithered patterns are used to compute the wrapped phase; and the average intensity is combined with two additional binary patterns to determine fringe order pixel by pixel in phase domain. The wrapped phase is temporarily unwrapped point by point by referring to the fringe order. We further developed a computational framework to reduce random noise impact due to dithering, defocusing and random noise. Since only five binary fringe patterns are required to recover one 3D frame, extremely high speed 3D shape measurement can be achieved.  For example, we developed a system that captures 2D images at 3,333Hz, and thus performs 3D shape measurement at 667 Hz.

"Method for large-range structured light system calibration," Appl. Opt., (2016)

[91] Y. An, T. Bell, B. Li, J. Xu and S. Zhang, "Method for large range structured light system calibration", Appl. Opt., 55(33), 9563-9572 (2016); doi:10.1364/AO.55.009563

Structured light system calibration often requires the usage of a calibration target with a similar size as the field of view (FOV), which brings challenges to large range structured light system calibration since fabricating large calibration targets is difficult and expensive. This paper presents a large range system calibration method that does not need a large calibration target. The proposed  method includes two stages: 1) accurately calibrate intrinsics  (i.e. focal lengths, and principle points) at a near range where both the camera and projector are out of focus; and 2) calibrate the extrinsic parameters (translation and rotation) from camera to projector with the assistance of a low-accuracy large range 3D sensor (e.g., Microsoft Kinect). We have developed a large-scale 3D shape measurement system with a FOV of (1120 × 1900 × 1000) mm^3. Experiments demonstrate our system can achieve measurement accuracy as high as 0.07 mm with a standard deviation of 0.80 mm by measuring a 304.8 mm diameter sphere. As a comparison, Kinect V2 only achieved mean error of 0.80 mm with a standard deviation of 3.41 mm for the FOV of measurement.

"High-accuracy, high-speed 3D structured light imaging techniques and potential applications to intelligent robotics," Int. J. Intell. Robot. Applic. (2016)

[90] B. Li, Y. An, D. Cappelleri, J. Xu and S. Zhang, "High-accuracy, high-speed 3D structured light imaging techniques and potential applications to intelligent robotics," Int. J. Intell. Robot. Applic. 1(1), 86–103, (2016).

Abstract

This paper presents some of the high-accuracy and high-speed structured light 3D imaging methods developed in the optical metrology community. These advanced 3D optical imaging technologies could substantially benefit the intelligent robotics community as another sensing tool. This paper mainly focuses on one special 3D imaging technique: digital fringe projection (DFP) method because of its numerous advantageous features comparing to other 3D optical imaging methods in terms of accuracy, resolution, speed, and flexibility. We will discuss technologies that enabled 3D data acquisition, reconstruction, and display at 30 Hz or higher with over 300,000 measurement points per frame. This paper intends to introduce the DFP technologies to the intelligent robotics community, and casts our perspectives on potential applications that such sensing methods could be of value.

Motion induced error reduction by combining Fourier transform profilometry with phase-shifting profilometry, Opt. Express, (2016)

[88] B. Li, Z. Liu and S. Zhang, "Motion induced error reduction by combining Fourier transform profilometry with phase-shifting profilometry," Opt. Express 24(20), 23289-23303 2016; doi: 10.1364/OE.24.023289

We propose a hybrid computational framework to reduce motion induced measurement error by combining the Fourier transform profilometry (FTP) and phase-shifting profilometry (PSP). The proposed method is composed of three major steps: Step 1 is to extract continuous relative phase maps for each isolated object with single-shot FTP method and spatial phase unwrapping; Step 2 is to obtain an absolute phase map of the entire scene using PSP method, albeit motion induced errors exist on the extracted absolute phase map; and Step 3 is to shift the continuous relative phase maps from Step 1 to generate final absolute phase maps for each isolated object by referring to the absolute phase map with error from Step 2. Experiments demonstrate the success of the proposed computational framework for measuring multiple isolated rapidly moving objects.

"Pixel-wise absolute phase unwrapping using geometric constraints of structured light system," Opt. Express, (2016)

[87] Y. An, J. -S. Hyun, and S. Zhang, "Pixel-wise absolute phase unwrapping using geometric constraints of structured light system", Opt. Express, 24(15), 18445-18459, 2016; doi: 10.1364/OE.24.018445

This paper presents a method to unwrap phase pixel by pixel by solely using geometric constraints of the structured light system without requiring additional image acquisition or  another camera. Specifically, an artificial absolute phase map, Φ_{min},  at a given virtual depth plane z = z_{min}, is created from geometric constraints of the calibrated structured light system; the wrapped phase is pixel-by-pixel unwrapped by referring to Φ_{min}. Since Φ_{min} is defined in the projector space, the unwrapped phase obtained from this method is absolute for each pixel.  Experimental results demonstrate the success of this proposed novel absolute phase unwrapping method.

"High-resolution, real-time simultaneous 3D surface geometry and temperature measurement," Opt. Express, (2016);

[85] Y. An and S. Zhang, "High-resolution, real-time simultaneous 3D surface geometry and temperature measurement," Opt. Express, 24(13), 14552-14563, 2016; doi: 10.1364/OE.24.014552

This paper presents a method to simultaneously measure three-dimensional (3D) surface geometry and temperature in real time. Specifically, we developed 1) a holistic approach to calibrate both a structured light system and a thermal camera under exactly the same world coordinate system even though these two sensors do not share the same wavelength; and 2) a computational framework to determine the sub-pixel corresponding temperature for each 3D point as well as discard those occluded points. Since the thermal 2D imaging and 3D visible imaging systems do not share the same spectrum of light, they can perform sensing simultaneously in real time: we developed a hardware system that can achieve real-time 3D geometry and temperature measurement at 26 Hz with 768 X 960 points per frame.

 

"Microscopic structured light 3D profilometry: binary defocusing technique VS sinusoidal fringe projection," Opt. Laser Eng., (2016)

[92] B. Li and S. Zhang, "Microscopic structured light 3D profilometry: binary defocusing technique VS sinusoidal fringe projection, " Opt. Laser Eng. 96, 117–123, (2017); doi: 10.1016/j.optlaseng.2016.06.009

Abstract

This paper compares the binary defocusing technique with conventional sinusoidal fringe projection under two different 3D microscopic profilometry systems: 1) both camera and projector use telecentric lenses, and 2) only camera uses a telecentric lens. Our simulation and experiments found that the binary defocusing technique is superior to the traditional sinusoidal fringe projection method by improving the measurement resolution approximately 19%. Finally, by taking the speed advantage of the binary defocusing technique, we presented a high-speed (500 Hz) and high-resolution (1600 X 1200) 3D microscopic profilometry system that could reach kHz.

"Single-shot absolute 3D shape measurement with Fourier transform profilometry", Appl. Opt., (2016)

[83] B. Li, Y. An and S. Zhang, "Single-shot absolute 3D shape measurement with Fourier transform profilometry," Appl. Opt., 2016; (accepted)

Abstract

Fourier transform profilometry (FTP) is one of the frequently adopted three-dimensional (3D) shape measurement methods owing to its nature of single-shot 3D shape recovery, yet it is challenging to retrieve the absolute phase map solely from one single grayscale fringe image. This paper presents a computational framework that overcomes this limitation of FTP with digital fringe projection (DFP). By using geometric constraints, an absolute phase map can be retrieved point-by-point from one single grayscale fringe image. Experiments demonstrate the success of our proposed framework with single-shot absolute 3D shape measurement capability.