Week #3 #
Implemented MVP Features #
1. 3D Scene Reconstruction Pipeline (Mahmoud Mousatat) #
We have successfully established an end-to-end 3D scene reconstruction pipeline, converting images into structured, interactive 3D visualizations. The main pipeline stages are:
MASt3R-SLAM: Produces dense point clouds from image sequences.
Point Cloud Alignment: Ensures canonical coordinate system orientation via PCA.
SpatialLM: Utilizes spatially-aware large language models for semantic scene descriptions.
Web Frontend (Three.js/WebGL): Allows interactive visualization of point clouds and scene layouts.
Backend API (Flask REST API): Manages processing jobs and integrates all pipeline components.
2. Drone Simulation Environment (Ilvina Akhmetzianova) #
Implemented basic drone simulation environment using Unity, focusing on simplified physics for ease of control.
WebSockets connection established between Unity simulation and Python-based backend, facilitating command execution and data exchange.
Developed intuitive drone control APIs for easy integration with future agentic systems.
3. Drone Data Server (Alexander Rozanov) #
- Dockerized backend server setup with dynamic indexing (ELK stack-inspired), providing flexibility for evolving data schemas without additional code modifications.
4. Agentic Architecture (Nikita Sergeev) #
- Initial conceptual sketches completed for the architecture of the agentic system, identifying key interactions with backend and simulation components.
Demonstration of the Working MVP #
visualization of the point cloud - https://vkvideo.ru/video-230535967_456239019 Drone simulation - https://vkvideo.ru/video-230535967_456239018
Internal Demo #
Demonstrated a working integration between simulation and backend services.
Verified the full reconstruction pipeline from simulated drone-captured images to interactive 3D visualization.
Received feedback to refine point cloud scale consistency for better real-world applicability.
Weekly Commitments #
Individual Contributions #
Mahmoud Mousatat: Developed and optimized MASt3R-SLAM algorithm; refined point cloud alignment method ensuring canonical orientation.
Ilvina Akhmetzianova: Finalized drone simulation environment, including Unity-Python integration via WebSockets and control API enhancements.
Alexander Rozanov: Completed backend server with dynamic indexing and containerized deployment via Docker Compose; performed code reviews and merges.
Nikita Sergeev: Developed agentic architecture sketches, highlighting integration points for future AI-driven drone navigation and data collection.
Plan for Next Week #
Achieve full integration testing between backend services and the drone simulation environment.
Begin implementation of the agentic system architecture based on the initial sketches.
Resolve outstanding scaling issue in point cloud generation by integrating accelerometer data from drone simulation.
Confirmation of Code’s Operability #
We confirm that the code in the main branch:
Is in working condition.
Runs via docker-compose (as described in the
README.md
).