NOT Schrödinger's Box

Inspiration #

Ever had a problem with having too many items? Want to find something but forgot where you have stored it? We too face a similar problem. With such a large number of items, we literally have no idea where everything is. All our items has become Schrödinger’s items, we simply do not know if it exist anymore. NOT Schrödinger’s Box is our way of tackling this issue.

What it does #

NOT Schrödinger’s Box helps you keep track of what items are stored within the box and where your items are. This will help us organize our items and help us keep track of what we have. With a simple click of a button, we can know what items we have and where exactly they are located by our internal alarm system.

1. Identify your box by scanning the QR code. Your box stores its location, and description of the nature of the items
2. Add your item by scanning your item and add a description if you wish
3. As an authorized user on our mobile app, open the box via the app. Our server will trigger to unlock and open your respective box automatically.
4. Add your item in. Similarly close and lock the box within the app.

Security: Should the physical log on the box be tempered with, a photo-resistor detects unauthorized opening of the box, an alarm will be sounded

Searching for a particular item: Not sure where your item was stored and in which box? Search up the item and it checks against all your existing boxes, helping you easily derive where to look at.

“Find my box!”: Unable to find your box in your darkened store room or amongst the clutter in the house? Find my box function within the app will give you an audio cue by emitting from the alarm system within the box.

How we built it #

For the application front end, it is built in React-Native. For the Server, it is built on Python using Flask. For the Iot, it is built on Python using GPIO-Zero.

Server: Flask: For routing and connecting the mobile application to the hardware which is controlling the box.

Hardware: OLED I2C - display status and process of the box Servo Motor 1 - Arm to automate opening/closing of the box Servo Motor 2 - Act as a locking mechanism, not allowing unauthorized users from opening the item Photo-resistor - Detect unauthorized opening of box Buzzer - Alarm System

Others: Internal and External Pull up resistors, and capacitors for various components Touch Screen Monitor - For the configuring of the RPi Push button (for prototyping) - Place holder for input signals from server

Arduino (during prototype) - For efficient testing of components RPI - Main house where all the hardware was integrated in, translated the low language in Arduino to Python.

Challenges we ran into #

Hardware was very limited this year and harder to work on due to the team working in separate locations for bulk of the time. We had to adapt to using other motors.

Detecting unauthorized opening of our box. We initially planned to use our Ultrasonic sensor, however it was damaged and there was no other pieces available. We adapted our Ultrasonic application from a Grove - Ultrasonic Ranger and prototyped it on the arduino. However the same module was not compatible with the Rpi. So we had to scratch it and improvise with a Light Dependent Resistor (Photo-resistor) to detect the opening and closing.

It was challenging incorporating the Camera function into the application as we were only able to obtain the local file path of the captured images initially.

Sending images across the web It was challenging to find a way to send images to the server. Initially we did not know of any ways to send images through the web. After searching online for solutions, we came to realize that we can convert the image to base64 before sending it over to be converted back into an image.

Using the RPi and interacting with the server which is running flask. Due to the higher level networking that was required on the side of flask, experimentation was required before we manage to form the packet which allowed for the raspberry pi to communicate with the server effectively.

Image recognition issues. We initially want to make use of the images taken by the camera to help the user recognize the object and make use of the data to suggest possible names for the items. However, due to different image types, the camera images from the phone were unable to go through the neural network (Yolo v3) to obtain the results that we need. Thus it was eventually scrapped.

Issue coming up with a specification for the format of the data that we are going to use: https://docs.google.com/document/d/1neaY6C6585HwASiEkawPydXfGZ-318Xz29zKdx8HgC4/edit?usp=sharing We spent a lot of time thinking about what needs to be included inside the json file in order to better facilitate the transfer of data between the application to the server as well as the server to the raspberry pi

Accomplishments that we are proud of #

We managed to meaningfully create an IoT hack that has both hardware and software aspects in such a short period of time. Our application is very useful for people who have many things to store (e.g. logistics personnel) as it can greatly help them to systematically organize and easily locate their items.

What we learned #

“Challenges can be overcomed if you work together as a team” -Junhua “Hardware is hard to debug” -Wan Ting “Boxes never looked so sexy before” -Ryan “Ohhh my gawwwddd” -RuiEn

What’s next #

In the future, more boxes can be added together with online integration so that it can find all your boxes without it being restricted to a local area network.

Object detection smart features can be integrated into the application to automate the naming process of items. In this way, users are able to efficiently add items into boxes without having to manually name every item. We have developed this to some extent using YOLO but we removed it from the final product due to latency issues.

GPS system can be included as well to assist users in locating the exact location of their boxes. This proves especially useful if the boxes are stored in varying places in a large facility. Users would be able to save a significantly large amount of time locating their items.

What is it built with (software) #

• Python
• Flask
• React-Native

Links #

• Github