Portable Architectural Design Developing a Immersive Augmented Reality System for Architects

Abstract


Introduction
With the success of smart-phones AR-applications have received a lot of attention.One reason is their hardware design: Devices like cameras or GPS-receivers, formerly physically separated, are now well integrated.
One new approach is head-mounting a smart phone as an affordable head-mounted-display (HMD).Because all needed technologies are integrated, added components would allow users to mount a smart phone in front of their eyes and view images in proper focus.As with all AR-systems a challenging task is the implementation of user input devices.Most systems ignore the difficulties and focus on presentation only without interaction.Neither the static real world nor the virtual part of the mixed environment are susceptible to manipulation.Users have only limited access to the system, their main control being the movements to explore the world.Common input devices like keyboards or mice, especially if they require visual control, are not an option.
Those systems are still capable of only representational purposes and entertainment.Their input capabilities are limited and usually designed only to control the system.For entertainment purposes this might be sufficient, as part of the architectural design process it is not.Some means are needed to create and edit virtual models, even in an immersive environment.

Concepts of Head-Mounted-Displays
With a dual channel display system similar problems need to be addressed.Brightness and color values have to be adjusted and the updates depending on the refresh rate have to be synchronized.The resulting solutions tend to be complicated and, hence, expensive.

Dual channel system
The complete duplicate installation of cameras and displays for both eyes is an assumed necessity for a truly immersive and spatial Augmented Reality.Two cameras as input-, two displays as output-devices and the processing of data between input and output need to be synchronized at all stages.The two cameras have to deliver the exactly same amount of light in the same colors to the underlying image processing components.The two cameras implicit need to be corrected for potential flaws at all times.
With a dual channel display system similar problems need to be addressed.Brightness and color values have to be adjusted and the Diseño updates depending on the refresh rate have to be synchronized.The resulting solutions tend to be complicated and, hence, expensive.The system introduced in this paper, while still at an infant stage, tries to avoid these problems by dismissing the duplicate secondary physical channel while still supporting stereoscopic viewing.

Single Channel
Single channel systems avoid all these problems.With the ongoing increase of pixels per inch (ppi), now at more than 300 ppi on some devices, some makers -people who build their own hardware -proposed using such a display as an HMD for gaming and published their ideas on related websites [1].Since then this concept had gotten currency without any support of the phone-manufacturers (Fig. 1).
The idea is intriguingly simple: a pair of magnifying glasses, an enclosure with a head strap, the smart phone itself and maybe some mirrors or prism are all that is needed for an immersive experience anywhere.While in landscape mode, the touch screen of the smart phone is divided into two equally sized parts.The phone is encapsulated in a plastic or cardboard mask to minimize ambient light.The view divider, a small fin perpendicular to the screen, focuses the view of each eye to its designated screen.
Smart phones provide additional challenges: There is no means to add an additional camera similar to the inbuilt device to the phone.Instead the view of the camera itself needs to be separated into two distinguishable views.This could be a simple copy or a cloned screen, or, more advanced, single-lens stereoscopic.
This technique requires, that the optical beam is divided in two distinct beams, which are deviated.Logically they assume two views from different, but related eye-points as it would be a natural stereoscopic view with two eyes respectively.Single-lens stereoscopic viewing done well simply voids the need of a secondary similar optical system.The basic techniques are not new.Photographers have long experimented with lenses, mirrors and prisms.Dividing a video stream in two geometric parts is an uncommon technique, and hence the support of the hardware section is slim.The software developed here offers several possible combinations for mirroring one or both viewing channels and swapping them to test the physical solutions already at hand.
The additional hardware is still very simple and just handcrafted.Recent results demonstrated that, while conceptually there is no flaw the quality of our immediate results lacking.Requiring the right hardware is still an obstacle and developing it is only the secondary option of choices.

Input Device
Other problems occur due to the mounting of an all-in-one device into an encapsulated enclosure while the user responsible to handle the device is blindfolded.While optimistically some buttons maybe accessible, the heart of every smart phone, its touch-screen interface is useless.It is impossible to make touch inputs on the inside of an HMD.
Luckily modern smart phones provide all kinds of wireless interconnectivity.We evaluated other devices as well.The most suitable of these would be common wireless game controllers.Such devices are well established, almost every user knows how to operate them.Most of them provide one or more standard programming interfaces, and they are affordable.
Still they are not optimal.The first modification of the offthe-shelf game controller was an attached grommet to connect a simple neck strap.Otherwise the controller might get lost and the user would have to strip off the head set and search for it.Secondly they are complicated.It remains a challenging task to handle a pair of joysticks and a dozen of buttons without visual control.
use Normal Arial 9 for body text.Italics must be used only to emphasize names, technical terms, or words in other languages."Only direct quotes should be in quotation marks, Arial 9 (neither bold nor italics), and without numbering."Details about citations are described in the citation section.

Designing with an HMD
The process is divided in two parts, sketching on a map and modelling in the immersive three-dimensional environment.

User-Interface
Because the touch-interface of the smart phone is not accessible, and the complete underlying controls are useless special transparent menus were developed in order to give the user some control.The movements of the joystick selects a menu-item, the trigger button chooses the function and dismisses the menu.The user can perform simple tasks like skimming through a menu switching to a map view or changing the virtual model.
All menus are duplicated for both parts of the display.Some special functions were also assigned to special buttons of the controller, such as a method to take a screenshot.There were other challenges to overcome.Smart phones are not developed as HMD-devices.As consequence all toolbars are hidden, auxiliary functions like notifications, animations as results of movements or rotations and similar actions are disabled.Inside the application introduced here the device is, as a smart-phone, crippled.This is not an app which can be shown right out of the pocket.

Sketching on the Map
Displaying maps is kind of a standard tool for smart phones, because showing a map is an important option to provide some orientation.It was an easy choice to incorporate the technology into the system.Using such maps is also valuable tool during the evaluation and the initial design phases of the architectural design process.
Again, there are two similar maps on both part of the screen.The maps themselves are controlled with the joystick, one moves it, the other zooms in and out.All testing users accepted the functionality.

Different Map Sources and Styles
Although maps are a standard tool, there are many different sources and also different styles around.Common are regular maps, satellite images and hybridpresentations, Popular styles are watercolor or black and white, which looks very similar to planning documents.For distribution almost all map sources are based on tiles and related map-tile servers.In addition there are some tile-servers especially related to GIS-data.In order to cover these sources, the standard map display was enhanced to allow for different map sources and styles from the very same location.

Drawing
To create objects in an architectural scale, simple shapes like circle or square are selectable from a menu and can be placed on the map as overlays.While technically scaling and moving them is an easy task, handling these shapes with a game controller is a complex task for the user.
The analogical model behind the user interface is a crane.Shapes are picked up, moved around and finally dropped off under the control of joysticks and buttons.Scaling objects is solved in a similar way.One side is chosen and then only the geometrical position of the side changed.

Modelling
The simple shapes transfer to an even simpler modeling tool.As with similar AR-applications the model is connected directly to the sensory environment of the smart phone and can be watched from all sides by simple head movement.This has become standard in recent developments, especially on smart phones, and needs no further elaboration here.If virtual three-dimensional objects are in range and in the line of sight, they can be manipulated as well.Moving them in the three-dimensionsal presentation is almost the same as presentation on the map.Scaling is reduced to modifications of the height.The user-centric coordinate system of the real world combined with the limited capabilities of the user's interface does not allow for more at present.
Once the challenges of the single-screen and single-lens are met a more complete modeling software will be implemented.The whole process of augmentation, sensing, registration and compositions will be applied and some features added.Additional augmentations like presentations of floor plans can add more design capabilities, other than haptic input devices can add more functionality

Discussion
The development of the system is still at an infant stage.The first prototype was an off-the-shelf device made only for virtual realities.It was used to understand the different components of the system and to develop the first software, especially the correct division of the incoming video streams and the user interface.
It soon became apparent that the industrial mask would not meet our requirements, so a second system was built.Because it is provisional, it's made of cardboard so we can break it apart and rebuild it again (Fig. 14).In addition two optical systems were built, the single mirror option was an easy one, the four mirror option proved to be challenging.The results shown here (Fig. 9) are correct, but there is definitely a lack of imaging quality which we hope to address.In fact the system shown here comprises two prism and two mirrors (Fig. 8).

Conclusion
All components of this AR-System are already well-established.The split-screen is an idea spread by makers, the single-lens stereoscopic viewing is an old concept from photography, the utilization of the sensors' environment in modern smart phones is well known, and the combination of several hardware devices like controllers, displays and headphones in order provide and establish an immersive ambience is almost standard procedure.
The novelty lies in the assembling of all these components.As result a unique type of a low cost AR system using existing smart-phones and simply constructed additions is emerging under constant development.

Figure 1 :
Figure 1: Different users with the device.

Figure 2 :
Figure 2: Conceptual diagram of two different single lens viewer, a solution with 4 prism, and another solution with only one mirror.

Figure 3 :
Figure 3: Self made cardboard HMD and prism attachment.

Figure 4 :
Figure 4: Results of the single lens viewers, on the left the solution with only one mirror, on the right the solution with 4 prism.

Figure 5 :
Figure 5: Map on the dual display and with the user interface.

Figure 6 :
Figure 6: Four different map styles from the center of Colgne, Germany.

Figure 7 :
Figure 7: Prototype mask with camera feed and augmented sphere.