1. models become controllable in the HoloLens AR

1.    Introductions

1.1.         Summary

For decades, science fiction movies have told us that our future world will be glutted with virtual images and experiences. A lot of famous engineers and scientists had also pointed out that we are following a trend of getting into a digital world. What we are facing are a mixed reality and holographic future. Augmented Reality (AR) and Virtual Reality (VR) technologies will also be widely used in the future for no doubt. Even recently, AR and VR have already been gradually imported in the field of game production, education and business training. This is also the reason why I am interested in the topic of the application of AR or VR for problem-solving.

In Hong Kong, there is a common difficulty faced by the water heater manufacturing companies. Every time when someone ordered a water heater for testing whether it fits or not, no matter will they purchase, the workers have to keep transporting and carrying the heavy water heater to and fro the customers’ houses. This is an inconvenience practice for both the customers and related companies.

Imagine if the customers can do the whole testing and product choosing process with only one small device, time and the human resources spent could probably be saved.

In this final year project, I am planning to put this as a real application with the support of AR technology. I will try to use the platform of SolidWorks, Unity, Visual Studio, and HoloLens to handle the whole project. Up to the present, I have created the 3D image of a few water heater models. Afterwards, I will try to let the models become controllable in the HoloLens AR device.

1.2.         Background information

Before people buy a water heater and install it at home, there are two important factors should be considered.

 

Firstly, it is necessary to have a clear understanding of space that available for the buyers to place a water heater. After measuring the space that we have, they would choose the one that is “most suitable”.  Then, they would ask someone to lift it to their address and try to place it in the bathroom. However, a problem may be recognized is that this may not be the “true place” that they want to install the heater. The placing position may either be too high or too low and it is hard for them to press the button and adjust the water temperature. If the water heater is too thick, the person who uses it may not even have enough space to move when having a bath. Therefore, after making sure the “real size of space” that is really available, they have to choose a better water heater and ask people to come and install it again. This action is really an inconvenience.

 

Apart from the heater size, the personal interest of the user is also an unpredictable factor. For some people, water heaters are also a part of the house, its design and shape is also important. Sometimes, after the heater was placed at the certain location at home, they may realize the one they chosen in the shop is actually not the same as they imagined. Then, another heater will be chosen and the whole process will need to be repeated again.

 

The above two situations are actually not rare in the real life. Sometimes, it would also reduce the efficiency of the business transaction between the buyers and sellers. It will be better if we can have a way to prevent these clumsy and inconvenience activities.

 

1.3.         Objective

In this project, the stakeholders that would be included are the companies that sell water heaters and their potential customers.

 

For the customers, the objective of this project is to increase the efficiency when they choose their water heaters. Originally, after a customer who chooses a heater that is suitable, he may need to book a time with the responding company and wait for someone to come and transfer the water heater. If he wants to change another, then he may need to wait again for another booking timeslot. In light of this, my project design is helping the customers to reduce this inconvenience and reduce the purchasing time for buying a water heater.

 

For the water heater companies, this project can help them to minimize the resource distributed during the purchasing process.  When a company received a request for trying and testing water heater, they need to assign some human resource to get and deliver the product. Sometimes, they may also need someone with basic acknowledgment to introduce the project to their target customers. As a rule of thumb, a company may need a least three person to handle a single transaction. In my AR project application, companies are enabled to assign only one staff member with one AR device and finish the whole customer purchasing process, including product selecting, trying and testing.

 

2.    Methodology

2.1.         Image Creation: SolidWorks and Unity

SolidWorks

SolidWorks is a solid modeling computer-aided design and engineering program on Microsoft Windows. It enables people to create and design 3D models by starting with different 2D geometries, like lines, arcs, points, etc. Size of the models would also be defined under the addition of dimensions. In this project, SolidWorks would be used as a platform for me to create different water heater models that can be shown in the AR system afterward.

Unity

Unity is a popular multipurpose program for building 2D and 3D games and apps (Lee, 2012). The game developer always creates game by using the external drag-and-drop function and C# scripting base on the needs of different projects. It also supports a wide range of devices’ application, including Windows, IOS, Android, HoloLens, and many more. In this project, it would act as the bridge that connects the 3D models and HoloLens. Different functions would be applied inside the unity, so that the user can freely control the location and dimension of the “water heaters” in AR.

2.2.         Function Creation Support: HoloToolkit

HoloToolkit contains many components that are specifically used in Unity. It is an optional toolkit that can enhance the development experience of project designers, especially those junior designers. The most important thing in this project is that it can provide a helping hand and teach me how to create the settings of gesture and spatial mapping. With the support of this source, I can do some adjustments and further elaborate so that all the settings can fit my project designed model.

2.3.         AR Device: HoloLens

HoloLens is a pair of mixed reality smart glasses, containing six cameras on the headset, five for tracking the environment and one for recording video and images (Lee, 2012). It can bring people, objects and surrounding environments from both physical and digital worlds together. It enables people to experiences the interaction with holograms. They can visualize and work with the virtual 3D models as part of their life. This is also the reason that I choose HoloLens as the corresponding AR device in my project application. Through HoloLens, my project users can recognize a clear 3D image of the water heater. Then, they can do the trying and testing process so as the real-life application.

3.    Project Researches

Augmented Reality

Augmented Reality (AR) is a technology that allows computer-generated virtual imagery information to be overlaid onto a live direct or indirect real-world environment in real time (Azuma, 1997; Zhou, Duh, & Billinghurst, 2008). In Virtual Reality (VR), people would expect to experience a virtual environment that generated by a computer. Different from VR, in AR, people would extend some additional information and images in the real environment through those AR devices (Lee, 2012). It somehow means AR is also a bridge that can link up both real and virtual worlds. In school, AR technology is usually used as augmented books with guidance, especially for some phenomenon and theory-based subjects’ education, like biology, physics, astronomy, etc. In business, people would tend to apply AR technology in the field of product design, technology explanation and staff training. In order words, the application of AR gradually become a new trend in everyone’s daily life.

 

Holographic

The basic principle of holographic techniques is to split the laser beam, so the surface of the subject can be reflected towards a photosensitive surface (Kirkpatrick, 1996). The second part is to reflect the beam into an interference pattern, which can be recorded and captured as a hologram. In bygone days, people used to view holograms by projecting the beam through the photo-plate and reconstructing the original 3D image of the required subject. Until now, many technologies are developed to create more flexible mediums, like HoloLens. Images are enabled to float on any plate and even inside a full 360-degree circular environment.

HoloLens VS Other devices

 

Microsoft HoloLens

Oculus Rift

Samsung Gear VR

Sony PlayStation VR

Zeiss VR One

1. Has Gaze And Eye Tracking Capabilities

?

?

?

?

?

2. Has A Socket For A 3.5 mm Audio Jack

?

?

?

?

?

3. Has A Display

?

?

?

?

?

4. Has A Gyroscope

?

?

?

?

?

5. Can Track Head Movement

?

?

?

?

?

6. Has Position Tracking

?

?

?

?

?

7. Has Gesture Control

?

?

?

?

?

8. Has Integrated Audio

?

?

?

?

?

9. Has An Accelerometer

?

?

?

?

?

10. Can Be Used Wirelessly

?

?

?

?

?

(Retrieved from Versu.com)

4.    Progress

4.1.         Current Progress

Design Project Topic

After recognizing the need of Towngas for handling the inconvenience problem of water heater purchasing, I was immediately assigned a final year project topic about how to solve the problem by using the AR technology. Then, based on my previous experience of using the HoloLens, I finally decided it as my major platform of the AR project.

Create 3D models

In order to let this project become more practical in the real situation, I had searched the real water models from the official website of Towngas. After collecting the some related information, including product design and dimensions of the water heaters, I had chosen some models with distinct difference. By following the given information, all the models are created in the scale of 1:1 through SolidWorks.

Create visible models in HoloLens

Then, the models were created and it is necessary to let it become visible in Hololens. Since the original 3D models created by SolidWorks could not directly be inserted in Unity, the file format should be changed so as to fit the format standard used in Unity. With the support from an online convertor, the water heater 3D models were successfully transformed from “.SLDPRT” files to “.3DS” files. After adjusting the model scaled inside the unity, and I could connect the program with HoloLens through Visual Studios. Meanwhile, 3D images were successfully shown and mixed with the reality in the screen of HoloLens.

 

Recently, I am finding the online tutorial resources and learning how to enable the users to interact with the water heater models shown in HoloLens.

 

4.2.         Obstacles to Progress

Lack of skills

Before this final year project, although I have already get in touch with the field of AR technology application, what I had tried were only the basic skills with tutorials to follow in step by step. I have never individually discovered the in depth theory of how to use the related platforms and create any models that can be shown in HoloLens without any supports. Besides, my skills of using programming language is still not enough for me to do the controlling part by myself. Therefore, I am still learning it and seeking some tutorials and resources that can help me to do this project. However, this may affect my efficiency of finishing the whole project.

Lack of resources

For the AR products that we can search in the market, most of them are used for either education, staffing training or game production. We can seldom find any related implementation of AR technology in the field of business operation. Especially in Hong Kong, AR is not that popular when comparing with other countries. This also means that the resources that I can follow is limited, while the project that I am doing is also a new input in business in certain extent.

Errors shown in the process

In the project until now, there are still a lot of small errors that I have to handle. Some of them could be solved immediately after recognizing the wrong parts. However, there were also some of them just happened with no reasons. It always made me confused and spent me a lot of time to solve it while I had to repeat the same process again and again. This shows that I am still not that familiar with this project topic and there are still a lot of things that I could learn and enhance.

4.3.         Future Progress Planning

Create moveable models in HoloLens

Since one of the this project is aimed to replace the traditional water heater trying and testing way, in order to let the users know whether the water heaters fits them or not, the models created should be moveable in HoloLens. Then, the customers can freely decide the locations that they want to insert the water heaters.

Create size adjustable models in HoloLens

In the real life application, the size of the water heaters is always fixed and un-adjustable due to the requirement of mass production. However, in the current industry, most of the other manufacturing products are following the trend of concretization while customers are able to order their own desired products. Therefore, I think it is also possible for the customers to adjust the size of the water heater within a certain range and what I have to do afterwards is to let it become possible in HoloLens.

Create selectable models in HoloLens

People used to purchase goods under a certain variety of choices. So as the water heater, customers should also be provided a list of products for them to choose. In the HoloLens, I will create a platform that allow users to change the “water heater” appeared in front of them freely.

5.    Conclusion

Augmented reality technology is a new topic for me to try and discover. Although there are still a lot problems and limits that I have to face and handle, I believe I can overcome them step by step. Hopefully, the AR model that I created at last can be useful and applicable in the real life practice, regarding the inconvenience problem solving of water heater trying and testing.

 

 

6.    Reference

Lee, Kangdon, 2012. Augmented Reality in Education and Training. TechTrends: Linking Research and Practice to Improve Learning, 56(2), pp.13–21.

Azuma, R. T. (1997). A survey of augmented reality. Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355-385. Cambridge, MA: The MIT Press. Augmented reality. (n.d.). Retrieved from https://www.ssatrust.org.uk/achievement/future/pages/AugmentedReality.aspx

Diane Kirkpatrick, 1996. Hologram. Grove Art Online, pp.Grove Art Online.

Cornell College, 2009. The 3D Revolution. Holmarc.com. Retrieved from http://www.the3drevolution.com/3dtv.html

Versus, 2017. Microsoft HoloLens vs Oculus Rift vs Samsung Gear VR vs Sony PlayStation VR vs Zeiss VR One: 18 facts in comparison. Versus.com. Retrieved from https://versus.com/en/microsoft-hololens-vs-oculus-rift-vs-samsung-gear-vr-vs-sony-playstation-vr-vs-zeiss-vr-one