3D Imaging News:
A large study comparing 2D and 3D mammograms for detecting breast cancer found that 3D mammograms (digital breast tomosynthesis) were significantly more effective than 2D mammograms. The study analyzed healthcare data from over one million women and found that 3D mammograms had a higher cancer detection rate and delivered lower false positive and false negative results compared to 2D mammograms. The researchers also noted that 3D mammograms worked similarly well for women with dense and non-dense breasts. While 3D mammograms are generally preferable, there may be rare situations where 2D mammograms are more suitable.
OpenAI has developed a new tool called Shap-E, which is a text-to-3D model generator. The tool simplifies the process of creating 3D assets and has the potential to disrupt industries like architecture, interior design, and gaming. Users can input a text prompt, and Shap-E will generate a corresponding 3D model. While still in the early stages of research and development, Shap-E shows promise in creating complex and diverse 3D assets quickly. Compared to OpenAI's previous 3D modeling tool, Point-E, Shap-E produces renderings with softer edges, clearer shadows, and less pixelation. Although the results are not yet on par with industry competitors, the speed at which Shap-E generates images is impressive, taking just 13 seconds from a text prompt. OpenAI continues to refine Shap-E and may collaborate with larger datasets to improve the quality further.
Research:
Researchers from University of California focused on the design of diffusers for single-shot 3D microscopy systems and the optimization of reconstruction algorithms. Here are two main findings from the research:
- Differentiable Optimization of Diffuser Surface Shape: The researchers propose a physics-based learning method to jointly optimize the diffuser design and reconstruction algorithm parameters. By using a differentiable forward model of single-shot 3D microscopy and an invertible and differentiable reconstruction algorithm, they enable the optimization of both the diffuser surface shape and the reconstruction parameters. This approach allows for the use of memory-efficient backpropagation and improves the reconstruction quality compared to previous heuristic designs.
- Memory-Efficient Backpropagation for 3D Reconstruction: To tackle the memory limitations of the 3D problem, the researchers employ memory-efficient backpropagation techniques. By utilizing forward checkpointing and backward Euler steps with a fixed point method, they can calculate the necessary gradients while fitting the problem into GPU memory. This enables the training of large-scale 3D volumes using a single GPU and significantly reduces the memory requirements.
Overall, the research introduces a physics-based learning pipeline for optimizing diffuser design and reconstruction algorithms in single-shot 3D microscopy. The approach improves reconstruction quality and addresses the memory limitations associated with large-scale 3D volumes.
Researchers proposed a novel method for reconstructing 3D surfaces using a multi-beam imaging sonar. The main findings of the research can be summarized as follows:
- Volumetric Acoustic Projection: The researchers developed a volumetric method that integrates the intensities measured by the imaging sonar from different viewpoints into a 3D grid. Each cell in the grid stores a feature vector representing the mean intensity for a specific range of viewpoints. By training convolutional neural networks using sparse range measurements as ground truth information, they were able to predict the signed distance and direction to the nearest surface for each cell. This approach enables dense 3D reconstruction even with a limited set of viewpoints.
- Truncated Signed Distance Field (TSDF) and Mesh Reconstruction: The predicted signed distances are projected into a truncated signed distance field (TSDF) along the predicted directions. The TSDF is then used to render a polygon mesh using the marching cubes algorithm. This allows for the creation of a detailed 3D mesh representing the reconstructed surfaces. The method was evaluated using real-world datasets and demonstrated promising results in achieving dense 3D reconstructions.
Overall, the research introduces a new approach for 3D imaging sonar reconstruction that overcomes the limitations of traditional methods. It leverages volumetric acoustic projection, neural networks, and TSDF-based reconstruction to achieve dense and accurate 3D surface reconstruction, which is particularly valuable for underwater environments where optical cameras may be limited.