Aleksandr Setkov took part in the art & science project La lumière ne s’arrête pas là developed at the neticLab (http://www.neticlab.org/) on the Paris-Sud University campus. The driving idea of La lumière ne s’arrête pas là is to represent to visitors the dependencies of the space-time metric with gravity by means of immersive video and sound projections. The installation is a mobile cybernetic prototype, which consists of a Lorentz room and a number of projectors set up inside and outside. Visitors can slide in one of the seven chambers attached to the room to explore the inside immersive and interactive projection. When visitors face shows up in the room and moves around, inferred perturbations are calculated to corresspondinigly modify the space-time projection. The videoprojectors outside cover the Lorentz room environment and surrounding, thus inviting visitors to come closer and to slide in the room.
The objective of the doctoral mission was to facilitate the work of artists that create content for the outside projection. Technically, the task consisted in the development of calibration and 3D reconstruction algorithms for the installation. Calibration was required for twin camera-projector systems set up in the installation, and 3D reconstruction could provide plane-wise segmentation to automate the matching process between the projection and the scene geometry.
The calibration algorithms were developed in collaboration with Franck Bimbard, assistant professor at LIMSI-CNRS. The calibration was performed by means of a printed checkerboard. The process therefore was split into two steps. The first one was the detection of the checkerboard corners and their matching to the real preliminarily-computed distances. The second step aimed at computing intrinsic camera and projector parameters.
To achieve 3D reconstruction, widely used structured-light techniques were chosen. First, the process commenced by a sequential projection of binary patterns on the scene and their acquisition by the camera in order to compute the correspondences between each projected and captured pixels. Next, two sets of matched projected and captured pixels were decimated to reduce the total size and the computation time. They were finally provided to the 3D reconstruction function along with the intrinsic device parameters.
Concerning the implementation, an available structured-light source code was taken and modified to fit in the project. Franck Bimbard developed a 3D reconstruction method. Since imprecisions in camera and projection calibration take place and represent one of the obstcles, experiments on the real installation have not yet been finished.
Detected corner points of a physical and a projected checkerboards.
Top-left and top-right figures – Samples of structured-light pattern projection on a surface; bottom-left and bottom-right – row and column decoding results.
with the help of Jean-Louis Marlats and the technical departments of Synchrotron SOLEIL