What we did: We designed and developed the camera optics, hardware and software, with board layout being handled by a local electronics partner. We handled industrial design, with prototypes 3-d printed for aesthetic evaluation and board fit. We managed the complete project through to production prototype stage liaising with suppliers and hardware manufacturers.
"We want to make a military grade ruggedized camera which produces 50MP panoramas. It needs to be portable, small enough to carry in one hand with a touchscreen LCD for control and monitoring. Full spherical capture including the zenith and needs to run on battery too - plus simultaneous capture from all sensors, immediate stitching and colour management in-camera. Can you design and have a working model within a year"
The project - codename Pandromeda - sounds like a tall order. When our client decided that's what they needed, they came to emobix. Many would have passed on the opportunity based on the technical and scheduling pressures of the project, but we chose to take it on. It was a great project from a technical perspective, but certainly not without its risks and challenges...
What is Pandromeda ?
Pandromeda is a panoramic camera which captures spherical images (almost the whole field of view in horizontal and vertical planes). The target usage scenarios are varied, including surveillance and monitoring applications, and also for architectural tours, surveys etc.
Key to the design was the concept of ease of use; simple setup, acquisition and review of images was a must. Pandromeda support both on-device control through a touchscreen LCD panel and remote operation through a PC or tablet-based UI over wifi or wired connections.
A key USP of Pandromeda was the one-touch acquisition and image stitching in-camera. Quite a tall order for a mobile device! In addition to stitching we also developed advanced whitebalance and HDR processing to maximise image quality. Given the target usage scenarios we had to deal with moving scenes (vehicles, people) rather than nice, controlled environments like church interiors so we had to ensure that the images captured by the four lenses were simultaneous – that’s basically four cameras in one, with a lot more processing required!
In addition to the photographic features the device also needed to be ruggedized to cope with difficult operating environments on the move. The design accommodated physical strength through an aluminium chassis with consideration for water resistance and thermal dissipation – ensuring that we keep the electronics inside cool and dry.
Pandromeda's Evolution
The project brief was to develop a portable panoramic camera for a client. The target was very high spec – a 50 megapixel capture with a full 360 degree coverage in the horizontal plane and ‘as much as possible’ in the vertical plane. Our target was to ensure that we capture the zenith (directly above the camera). Combine this with simultaneous capture from all sensors and in-camera processing and stitching, battery or mains operation, high environmental tolerances and remote operation over wifi in a very compact portable case and you get a feel for how much of a challenged we had - that’s before we even consider the very compressed timeframe.
The camera was a leading edge product, developed from the ground up. Emobix was design lead on the project covering mechanical and industrial design, prototyping, electronics schematics & design, camera feature development, optical design and software, working with partners for parts of the electronics design, layout and manufacture.
The initial stages of the project involved fleshing out the product spec and a detailed analysis of optical requirements. Due to the fact that the maximum product dimensions had been fixed by the client we had a difficult task trying to come up with a solution where the physical optics worked within the confines of the compact case.
A significant amount of effort went into the design of custom lens mounts which allowed accurate alignment of the lens and sensors, combined with a housing which provided accurate relative alignment of the 4 lenses used in the design.
Various prototypes were built to test the conceptual arrangement and capture images using the lenses in a test rig. A number of materials and design variants were tested for the mounts in order to satisfy both the optical tolerance requirements and ease of assembly during manufacture and maintenance.
Once the optics and mounting arrangement had been validated it was possible to focus attention on the case design incorporating the lens mounting assembly. Case cosmetics were always kept in mind, but the initial focus was firmly on ensuring that the lens assembly and electronics would actually fit in the relatively small space we had to work with!
Work on the electronics continued with assessment of various potential CPUs which could meet the demanding performance, thermal and power requirements of Pandromeda.
Once the main CPU family was selected work began on selection of peripheral components, UI devices and ancillary processors. Manufacturer's development board was used to progress both the hardware and software development. A custom OS distribution was created to ensure that only the required device drivers and features were loaded to give as fast a boot time as we could manage. Touchscreen LCDs were assessed using simple Arduino boards and once the ease of use and accuracy was validated we built some prototype LCD boards which interfaced with the mainboard to allow us to gradually build the camera capability around the development boards. The same approach was taken for ancillary components such as GPS and motion sensors.
Selection of the best components is a difficult task, but making sure they all fit in the compact case dimensions with the lens assembly and battery is a completely different type of challenge. We roughed out the component sizes and locations then looked at how we could split the functionality onto multiple boards within the free area in the case, taking into account assembly/disassembly needs and user ergonomics such as siting of the external ports and touchscreen LCD.
We developed 3D models of the boards, case and components to both visualise the overall system and ensure appropriate clearances. Once the models were analysed and we had run through various scenarios with them it was time to move on and progress the electronics schematics & layout, as well as the case itself.
3D models are an invaluable tool, but there is nothing like having the real thing in your hands to get a feel for how well the design meets requirements and also whether the design ergonomics really work. We considered various approaches but in the end we decided to 3D print the model parts. This provided us with great feedback on what it would be like to use the camera in real life scenarios, and also enabled more productive discussion with the client on the aesthetics and usability of the device.
The electronic design progressed in parallel, with no shortage of challenges trying to shoehorn the LCD board, image processing, CPU, power and comms into a fairly small area whilst dealing with thermal and EM aspects of the design. Due to our compressed project timescales it was not feasible to cover a number of design iterations so we put significant effort into design validation and checking schematics against layout in order to minimise the need for respins or patches. We manufactured boards as designs were checked and approved, allowing them to be QA tested as soon as possible.
With more complex boards such as the main board with a BGA CPU package it can be fun trying to identify problems bringing up the boards, especially on the first-spin. Pandromeda was no exception; in general the first set of boards were exceptionally successful. A couple of boards did present some problems which were not immediately obvious; power and regulation were as expected and inspection of the board was satisfactory. Narrowing the issues down to RAM or CPU, it was obvious that normal testing would not be enough on its own. The boards were booked in for x-ray at a local manufacture and test facility. The scanner and results were impressive, allowing us to identify issues with shorting of the BGA soldering and a couple of the DDR3 RAM chips - cue some rework of the offending chips and all issues sorted.
Quite an impressive result for first-spin electronics on such a complex sytem; we were quite pleased with the results, justifying the stringent attention to detail on the schematic design, prototyping and QA of the board layout prior to manfacture. This was a risky but essential strategy given the tight deadlines for the project and the long turnaround time on respins and manufacture.
With the electronics up and running we were able to focus effort on the device firmware and remote tablet application. Fundamental RAM timings were calculated for the DDR3 RAM an basic kernel boot was completed before moving on to peripherals. Availability of WiFi drivers for the ARM Cortex had proven a particularly thorny issue, but we eventually found a workable solution and moved on to proper testing of the lens/sensors, stitching and implementation/tuning of whitebalance and tonemapping features.To do this we built a test rig to house the sensors and board stack - an exoskeleton of the case configuration, but allowing access for debugging/testing of board components in a live setup. This phase soaked up a reasonable amount of effort - probably more than anticipated - but provided the ability to field-test the final camera configuration.
We had aluminium cases milled for the client and assembled our first batch of production prototypes which are in use for sales and marketing by the client. The project has been transferred to the client as they have secured additional funding to build an in-house team to handle productisation and ongoing development of their IP. We wish them success in their venture.