Modular 3D Scanner Toolkit

> about

> roadmap

> process

> videos

about

Modular 3D Scanner Toolkit is a custom built software that allows to capture various objects, creatures and spaces as 3D models in near real-time. The aim of the toolkit is to make it easier to combine views from multiple depth cameras into a seamless pointcloud and to enable further experiments with 3D scanning things in motion. It is designed and developed by Studio Pointer* and was originally created as part of interactive art installation Planet of People, most recently exhibited at Venice Architecture Biennale 2021 where it was presented by the Lithuanian Space Agency (the Lithuanian Pavilion). After working on multiple iterations of the software, studio Pointer* decided that this tool can be useful for other artists and developers and that the best future for it is to be open source and out in the world.

# roadmap ● - implemented
● - to be implemented - [x] tested systems - [x] Linux 64bit >= 5.3.x - [x] Ubuntu 18.04 - [x] Ubuntu 20.04 - [x] Debian 10 - [x] Debian 11 - [x] Manjaro (Arch) - [x] GPU - [x] Nvidia 1070/1080/1080TI/3070 - [x] AMD Radeon 6600 - [x] devices - [x] Azure Kinect - [ ] Realsense D415/D435/etc - [x] functionality - [x] create or modify openFrameworks addons - [x] [openFrameworks](https://gitlab.com/pointerstudio/utils/openFrameworks) - [x] add cpp17 on linux - [x] [ofxPCL](https://gitlab.com/pointerstudio/utils/ofxPCL) - [x] installation script - [ ] examples - [x] [ofxDepthDevice](https://gitlab.com/pointerstudio/utils/ofxDepthDevice) - [x] Kinect Azure SDK wrapper and implementation - [ ] Realsense SDK wrapper and implementation - [x] multi-threaded depth streaming (per camera or per task) - [x] installation script - [x] examples - [x] [ofxCv*](https://gitlab.com/pointerstudio/utils/ofxCv) - [x] installation script - [x] add Aruco/chAruco - [x] marker pose detection - [x] examples - [x] [scanner](https://gitlab.com/pointerstudio/depthdevice/scanner) - [x] calibration - [x] rough manual calibration - [x] rough calibration with markers - [x] fine calibration by point cloud shape - [x] export camera poses - [x] json array per calibration method - [ ] openCV style yaml - [ ] Blender, Godot, Unity, UE4 transfomation nodes - [x] simultaneous capture of scan - [x] scan optimization - [x] calculation of normals - [x] automatized cleanup - [x] adjustable levels of detail (model resolutions) - [ ] textured mesh - [ ] internal human skeleton detection - [x] Wavefront obj output - [ ] assimp - [x] [human skeleton detection](https://gitlab.com/pointerstudio/skeletonfrom3dscan) - [x] openpose docker - [x] python script skeleton from wavefront obj

# process The toolkit can be used either as a scanner application for Linux, or an openFramworks plugin. We are aiming to provide a precompiled package of the application soon, but for now you would need to compile it yourself and some knowledge of openFrameworks and Linux is useful. ## Using the openFrameworks plugin: You'll find the plugin and a short readme with installation instructions here: [https://gitlab.com/pointerstudio/utils/ofxdepthdevice](https://gitlab.com/pointerstudio/utils/ofxdepthdevice) It has been successfully tested with various distributions. The plugin is in constant development, but can be considered stable. Our maximum non-stop uptime was 221 days with six cameras on one machine. Theoretically it should also run on Windows, but we have not tested it. ## Using the scanner application for Linux You'll find the application and instructions to build it here: [https://gitlab.com/pointerstudio/depthdevice/scanner](https://gitlab.com/pointerstudio/depthdevice/scanner) This application implements multiple calibration techniques and is able to produce a merged and optimized pointcloud model. At the moment of writing, the usage is rather functional and not user friendly. Any suggestions for improvement are welcome.

Special thanks to Fred Rodrigues

This project is made possible thanks to support from Stimuleringsfonds