"Friender" - GP2: Phone Sensor Unlock
Developed by Ishrat Naba, Anh Nguyen, and Christian Salvador
Description
Are you an up and coming mobile phone startup looking for an unlock method with a more personal touch but tired of the same gimmick almost every modern phone unlock uses? Well in this project, Ishrat, Anh and I were tasked to develop an unlock system that utilizes the many sensors of modern day phones instead of the usual touch screen for phone unlocking. To start this off, our group decided to sketch out 10 different ideas for unlocking a phone using sensors available on recent iPhones.
After a round of 10 concept sketches and 10 refined sketches, which will be discussed in the sections below in more detail below, we decided to go with an idea that mimics both the style and interaction of Tinder but in the context of a phone unlock screen. Furthermore, instead of swiping left and right, we opted to use the acceleration sensor on the iPhone to mimic the interaction of sliding or flicking polaroid images off of a stack. Tilting the phone left would discard a polaroid, and tilting right would accept a polaroid image. The polaroid images were initially intended to have been real people but it can technically be used on images of anything that a person has a personal relationship or attachment with.
The application can be accessed using this URL: https://friender-two.vercel.app/
You will first be greeted by a regular looking iPhone lockscreen with instructions to swipe up to initiate the unlock screen like the one shown below. The reason for this is to avoid accidental initialization of the unlock system and tells the phone when to start listening to device motion.
After swiping up, you will then see a stack of polaroid images and dots at the bottom of the screen like the image below. For the purposes of the current live demo, the unlock sequence is Naruto > Frieren > Alan > Gojo (Student). With an iPhone, a user can tilt the phone left to discard a polaroid, or tilt the phone right to accept a polaroid as one of the correct ones. You will see that tilting left will not update the dots, but tilting right will. This gives visual feedback to the user on how far they have progressed with the unlock. If a user manages to discard all the polaroids, the stack will reshuffle and the user will have to start over again. The same happens when a user accepts the incorrect sequence of polaroids. Furthermore, the dots will shake and text will appear to notify the user that they failed the unlock as another form of visual feedback.
Unlocking Screen - Click to enlarge
Once the user inputs the correct sequence of screen unlocks, they will be greeted with an animation of the unlock screen pulling itself up to reveal the app screen below!
In terms of security, someone trying to breach the phone will not only need to know which polaroids are correct but they will also need to know the correct order to select them. The order of displayed polaroids are always randomized upon reshuffle but the correct polaroids always appear within a set specific number of cards to prevent frustration from the user potentially taking too long to unlock their phone. Furthermore, there is nothing stopping the owner from selecting different images of the same person or object to make it more difficult to breach since a specific image needs to be selected regardless of who or what the subject is.
10 Concept Sketches
For my 10 concept sketches, I began sketching ideas that utilized at least one of the many sensors available in an iPhone as all three of us in the group own one as our main phone. In this section I will be providing the sketches I have drawn below. For sketches 1, 4, 9, and 10 specifically, I will be providing descriptions of these ones, as these were either one of my favorites, picked as favorites by all of my group members or inspired some aspect of the interaction within our final design.
Sketch #1: This idea utilized the gyroscope available in an iPhone and played around the idea of mixing chemicals in a specific order to unlock the phone. This has a similar interaction in our final design in that just like how tilting mimics tilting a real vial to pour out liquid, the tilting/shaking in either direction would slide/flick a polaroid in the same direction.
Sketches 1, 2, and 3 - Click to enlarge
Sketch #4: An unlock system that involved the gyoscope and accelerometer to customize a person's outfit to unlock the phone. This idea specifically was well liked within the group as it had a more personal touch to unlocking the phone. This had also given a bit of an inspiration in incorporating something more personal when I went to draw my refined sketches.
Sketch #10: This idea utilized the camera as a way to input morse code to unlock the phone. This idea was also well liked within the group due to both simplicity and clever usage of using morse code with the camera.
Sketches 4, 5, and 10 - Click to enlarge
Sketch #9: This unlock would utilize both the accelerometer and gyroscope together to simulate a boxing match, where a user would have to input a spcific set of combos against a virtual opponent to unlock the phone. I primarily enjoyed how fun of an idea and concept it was.
Sketches 6, 7, 8, and 9 - Click to enlarge
Refined Sketches
In this section, I will showcase 3 of my 10 refined sketches as these were important in how we came to the idea of our final design. Two of them also involve alternate designs which I personally thought was clever to include.
After discussing each of our own 10 concept sketches, we decided to refine the ideas that involved "Color", "Shaking" and "Code Tapping". Ultimately, "Shaking" was the one that ended up being involved the most in our final design.
For sketch #4: This was a refinement on the idea of using the camera as a way to tap. I had thought of a clever way to "slide to unlock" without stepping on the toes of Apple and their patent. This interaction utilized sliding the two cameras behind their most recent iPhones in an interaction similar to how Apple used to do it with their patent with the touchscreen. Of course, this was more of a sketch on interaction but had no thoughts on security.
For sketch #5: This utilized the idea of that camera slide interaction in a context inspired by Tinder, as we had discussed that idea during our discussions. This inspired me to use the camera slide interaction to mimic the swipes used in Tinder.
For sketch #6: This turned into more of a variation of sketch #5 in that it utilized shaking/tilting the phone to mimic the swipes. We ended up going with this interaction design as it felt the most comfortable and doable within the timeframe we were given.
Refined Sketches - Click to enlarge
Final Concept Sketch
This section includes a further refined final concept sketch drawn by me and was primarily used as the concept sketches for the final design of our unlock system.
Final Concept Sketch - Click to enlarge
Final Product Reflection
My Contribution
My main contributions to the project involved getting the actual unlock UI and logic implemented. This involved setting up how the stack of polaroids were displayed, how the visual feedback on progress, success and failure were displayed, the animations that played during the interactions, and the logic of the reshuffling and tracking of selected cards.
Team Reflection
I believe my team worked very hard even with the timeframe we were given. Unfortunately, many of us had difficulty in putting as much time as possible into the project due to conflicting schedules and unexpected events. However, my group managed to pull through and we got our main objectives and UI implemented in a way that we were happy with. In terms of contribution, while I did work on the UI and logic of the unlocking sequence and its respective page, Anh setup the React and Ionic project for us as well as the shaking interaction, and Ishrat put more work into the initialization of the unlocking system with the swiping interaction and initial lock screen. Overall, I think my group put in a lot of effort into the project despite our unfortunate situation with schedule conflicts.
Objective
I believe our final design of an unlock screen fits well within the objective of the assignment. We've developed an unlock system that utilizes the accelerometer on an iPhone that is used to interact with our unlock system. Furthermore, we have visual feedback on the progress, success and failure of unlocking the phone. Checkboxes at the bottom of the screen track the progress of how many polaroids have been selected. The lockscreen pulls itself up upon success, revealing the user's homescreen underneath. Finally, failure results in the checkboxes shaking themselves clear, with text notifying the user that the sequence was incorrect and the stack of polaroids reshuffle themselves. In terms of accidental activation, we implemented a swipe up screen to intitialize the sequence. Furthermore, the phone must be turned in both directions to accept and reject polaroids, making it difficult for accidental activation in someone's pocket.
What Could Have Been
If time had permitted it and there wasn't any conflicts in scheduling, I would have loved to incorporate the camera swiping interaction from my refined sketches. I would have also loved to incoporate interactions that would allow a user to view bios of the images for a more creative and interactive design. I feel like our design lacked a bit in sensor interaction but we tried making up for it in the creativity of our idea. More physical feedback like audio and haptics could have also given more life to our design but unfortunately we just did not have enough time to incorporate that.
What Worked and What Didn't
The one thing that I could think of that didn't work was the development on an iPhone. Although all of us owned an iPhone and at least only one of us had an Android phone available, it made testing and debugging a nightmare. Interactions and logic that seemed to have work on Chrome seemed to had bugged out when run on an iPhone. Furthermore, iPhones require the deployment of an app in order to even receive permissions to use sensors which made development slow and difficult on this platform, especially with the time constraint we were faced with.
However, in terms of what did work, I believe the concept of our idea worked really well with what we wanted and were able to achieve due to our unique situation. I feel like the simplicity of our sensor usage was made up for with the interesting design of our unlock system.