Idea
Hey! My name is Vitaly, I am a developer and co-founder of a startup BeWarned... Since childhood, I have very weak hearing, and my wife cannot hear at all, for a long time I wanted to start developing applications and software for the deaf and hard of hearing. All the problems they face are familiar to me.
Me and my wife Natalia
Previously, I was engaged in the development of software for civil airlines in Ukraine, I created the SIMO augmented reality application. I have always been interested in everything related to technology. A couple of years ago, I met Sergei Malyukov, now our CEO, through a teacher at the Kiev Polytechnic Institute, with whom I once studied.
The idea of creating an application for deaf and hard of hearing people came to Sergey in 2013. Once he was walking down the street in the evening, and a car started honking in his back, Sergei quickly jumped aside, but the couple walking nearby did not budge. He shouted after them to move away, but they stayed where they were. Soon Sergey realized that it was a pair of deaf people, they communicated with each other in sign language. Sergey was convinced that there were applications that would help him recognize dangerous sounds. After analyzing existing applications and gadgets for deaf people and not finding a single solution that helps to recognize dangerous sounds, he set out to make such an application. In the process of work, several partners joined him and now BeWarned is self-investing by the founders of the startup.
As soon as I heard about the project and saw the developments, I was instantly carried away by the idea and soon became a co-founder. At the moment I am engaged in the technical development of a startup.
Implementation
In 2014, the first prototype of the BeWarned app was released with the Sound Monitor feature, which made it possible to recognize dangerous sounds in real time. This was the first small success. But that was only the beginning.
Since childhood, I liked technology, I often assembled or disassembled something, I wanted to see what would come of it
From hundreds of emails and reviews, it was clear that deaf people need more than an app that recognizes dangerous sounds. The deaf had a thousand unresolved questions that could be resolved. Based on these letters and feedback, the BeWarned platform was created, which united 4 technical assistants for the deaf: Sound Monitor, Connect, Emergency Call and Dance.
Sound Monitor solves the security problem by detecting dangerous sounds (screaming, car horn, dog barking, siren). In case of danger, the application warns the user by vibration and flashing of the flash on the smartphone. Connect helps the deaf in communicating with others - it converts text to speech and vice versa. Dance allows you to experience music tracks by converting them into vibration, light signals and pulsation of the visual equalizer. Emergency Call helps the user to call hearing loved ones for help when he is in a dangerous situation.
More videos can be found on our channel
In the summer of 2016, we conducted beta testing of the platform in the United States. As a result, 88.9% of users reported that they would recommend the app to their friends.
The most difficult part of developing BeWarned was Sound Monitor, which we spent several years developing. The program for recognizing dangerous sounds is based on neural networks. We tried two types of neural networks: the classic multilayer perceptron and the convolutional neural network. But then we decided to try the hidden Markov model, it was the best fit for our tasks.
In order to create a database on which the algorithm will be based, we ourselves recorded many variants of one type of dangerous sound, for example, a car signal, on the basis of which the algorithm will recognize incoming sounds and determine them as belonging to a certain type. However, in the course of work, we had a problem that the recognition algorithm attributed all incoming sounds only to the existing categories: a car signal, a shout, the sound of a police siren and a barking dog. For example, if someone was laughing, then the program would recognize it as belonging to one of these categories. To solve this problem, we had to segment sounds, for example, by frequencies and other characteristics. As a result, it was possible to introduce an additional indefinite kind of sound.
Some time after the release of the BeWarned platform, we decided to make important changes in order to improve usability: we radically simplified everything and adapted it to the perception of deaf people. This is why we re-released the apps separately.
In December, BW Dance (Android, iOS) was released, which turned from a function of the BeWarned platform into an independent application. BW Dance allows deaf and hard of hearing people to dance to their favorite songs. The user adds his favorite songs to the playlist, and the application converts the music to vibrations, flashes and animations on the screen. The user can also feel the music through the visual player. BW Dance was positively received by users both in Ukraine and abroad.
After that, we released Connect by Bewarned (Android, iOS), an application that allows deaf and hard of hearing people to communicate with hearing people. In order to start communicating with one or more people in the application, you just need to press the START button and let the interlocutor speak, after that the application converts the speech into a text message that a deaf person can read. The app also converts text to speech, a deaf person needs to write what he wants to say. In addition, we have developed a set of templates that simulate simple, repetitive situations, such as ordering coffee at a coffee shop. With these templates, a deaf person can quickly communicate something to the hearer. We also provided the ability to create additional templates for the personal purposes of each deaf person.
results
In Ukraine, journalists began to actively write to us, stories about us were filmed by STB, 1 + 1, Channel 5 and others. We saw that people care about the deaf, and we also made sure that focusing on simplicity and usability was a step in the right direction. We realized that we were on the right track.
My wife and I talk to journalists from Channel 5
We launched Connect by Bewarned at Product Hunt on January 30 and entered the TOP 5 of the day and also got into the official mailing list. In the comments on our product at Product Hunt, we received offers to help develop localizations in French and even Farsi. Together with the already existing four localizations - Spanish, Russian, Ukrainian, and English, we will be able to get closer to our global goal: to make life easier for people who are deaf.
I am convinced that IT people should use the skills given to them for the benefit of other people. Our small but successful experience shows that new applications can significantly change the lives of other people. That is why we are going to continue to work on the idea embodied in Connect by Bewarned: overcoming the barriers in communication between the deaf and the hearing, there are about 400 million of them in the world.
BeWarned team from left to right: Me, Evgeny, Evgeniya, Ivan, Julia, Sergey, Andrey, Ilona, Ira, Anton, Andrey (now there are even more of us than in this photo :))
People like me often have problems when they need to communicate with a hearing person: a policeman, a doctor, a banker. We are legally guaranteed equal rights and access to services and services on an equal basis with hearing people. But in reality, our capabilities are significantly limited. That is why we decided to tackle a truly global task: creating a global electronic infrastructure that will facilitate the access of deaf people to all services and services that are available to hearing people: banks, hospitals, taxis, etc. At the moment we are working on exactly this task.
Two female students from Beijing Beihai University have developed an app to help deaf people in China. The application was named Shouyin, which literally means “the voice of the hands”. It allows you to translate the signs of the language of the deaf into speech in real time.
The application works in conjunction with a smart bracelet and at the moment can recognize more than 200 characters, each character has been recorded a thousand times from different people to improve the accuracy of the translation. Now the recognition rate is 95%.
Young Nana Wang, one of the developers, said that she was inspired by a deaf-mute friend with whom she could only communicate through messages on her mobile phone. In China, approximately 5% of the population, or 70 million people, are deaf. The application will undoubtedly be a great success. The official release of the application is scheduled for the end of May. Similar programs already exist in other countries of the world. For example, MotionSavvy's Uni and Google Chats' deaf and dumb translator.
Technologies are developing at a rapid pace today, including in social spheres and in medicine. They help not only to cope with various kinds of ailments, but also to better understand your neighbor, even if his perception of the world differs from yours. A special application will help you to talk with a deaf-mute person, while VR technologies can help, for example, with understanding autism, and therefore bring you closer to those who suffer from this disease.
Various studies have shown that over 99% of people have heard of autism, but only 16% really understand what this mental condition means. And according to a UK survey, more than 25% of people with autism have ever asked to leave a public place like a restaurant or shop when they saw symptoms of their illness.
To tackle this disinformation problem, the British National Autistic Society decided to create VR movie that would help all people know what it is like to live with such a diagnosis. GmbH has written about this.
This virtual video will take you into the body of a young boy with autism and show the world through his eyes. In the movie, you walk into a shopping mall and experience the tremendous sensory load that he feels when he walks through this crowded place. Noise, an abundance of various objects, flowers, flickering of people, their voices - all this causes the boy to panic, which is very difficult for him to cope with.
Another Alzheimer's community has also released a VR movie that puts you in the shoes of someone with dementia. The movie "A Walk Through Dementia" will allow you to understand how some simple daily activities, such as making yourself a cup of tea, can become a real problem for the owner of the disease.
The main problem with such applications is that this niche is underdeveloped. In my search, I looked at over 30 applications, and almost all of them were designed for an English-speaking audience or people living in Europe or the United States. However, the six applications that are presented below are an exception and, I hope, will be useful to someone.
Miracle modus
Miracle Modus is an app created by an autistic developer. Judging by his words and reviews in the Play Market, the application really works. The principle of operation is simple and complex at the same time: Miracle Modus draws various lines and patterns and produces relaxing sounds that have a calming effect on people with autism.
Diabetes Log Book
An application that helps people with diabetes of the first and second degree. Here you can enter all the information about the food that you ate during the day. The complete report can then be exported to Excel to show to your healthcare professional.
Migraine buddy
Allows migraine sufferers to analyze their lifestyle and understand what factors lead to headaches. The app also tracks your sleep so you can see the correlation between migraines and sleep quality.
Bumps
Bumps helps blind people learn Braille. The application supports only English, so it is suitable only for those who want to learn to read not only in Russian. In addition to learning the alphabet, Bumps lets you enter different words and watch them write in Braille.
There is a similar application for Android - Braile Guide.
Spread Signs
Spread Signs has a huge library of gestures to help deaf and dumb people communicate. The application base includes more than a dozen languages, including Russian, Ukrainian and English. You can learn both gestures for individual letters and whole words.
Simple actions such as talking on the phone, going downstairs, or even determining the expiration date of milk can become difficult for people with disabilities. However, some of these problems can be successfully dealt with using a smartphone.
Introducing a selection of mobile apps that make life easier for people with disabilities.
Applications that recognize speech
Yandex.Talk
Yandex recently released an application that helps deaf and hard of hearing people communicate with acquaintances and strangers by translating spoken speech into text and vice versa.
The application can listen and recognize speech - everything that is said to you is displayed on the smartphone screen in the form of text, as well as pronounced the typed words aloud.
The app was invented and developed by a group of MIPT students and is powered by Yandex SpeechKit speech recognition technology.
RogerVoice
RogerVoice is an application that allows deaf people to talk to their interlocutors on the phone. The app uses speech recognition technology to convert voice to text so deaf people can read what the other person is telling them.
The application appeared thanks to a successful Kickstarter campaign - A beta version of the application is currently available and can be downloaded from the RogerVoice website.
An app that helps you hear
HearYouNow
With this application, hearing impaired users can adjust the sound according to the situation they are in - in a public place, in a meeting, etc. To do this, headphones are connected to the smartphone, through which sound is supplied with appropriate settings to optimize the background or foreground.
Developed by ExSilent, a Dutch hearing aid manufacturer, the app is aimed at people who can use hearing aids but are not yet ready for it.
Hello!
"Bread nourishes the body, and the book nourishes the mind" ...
Books are one of the most valuable treasures of modern man. Books appeared in ancient times and were very expensive (one book could be exchanged for a herd of cows!). In the modern world, books are available to everyone! Reading them, we become more literate, our outlook and ingenuity develops. And in general, we have not yet come up with a more perfect source of knowledge for transferring to each other!
With the development of computer technology (especially in the last 10 years), it has become possible not only to read books, but also to listen to them (that is, you will have a special program to read them, in a male or female voice). I would like to tell you about software tools for text scoring.
Possible recording problems
Before moving on to the list of programs, I would like to dwell on a common problem and consider cases when the program cannot read the text.
The fact is that there are voice engines, they can be of different standards: SAPI 4, SAPI 5 or Microsoft Speech Platform (in most programs for text reproduction there is a choice of this tool). So, it is logical that in addition to a program for reading by voice, an engine is needed (it will depend on which language you will be read in, in which voice: male or female, etc.).
Speech engines
The engines can be free or commercial (of course, commercial engines provide the best sound quality).
SAPI 4. Outdated versions of tools. It is not recommended to use outdated versions for modern PCs. Better to take a closer look at SAPI 5 or Microsoft Speech Platform.
SAPI 5. Modern speech engines, there are both free and paid. Dozens of SAPI 5 speech engines (with both female and male voices) can be found on the internet.
One of the best text-to-speech software. Allows your PC to read not only simple txt files, but also news, RSS, any web pages on the Internet, e-mail, etc.
In addition, it allows you to convert text to mp3 file (which you can then download to any phone or mp3 player and listen on the road, for example). Those. you can create audio books by yourself!
Voices of the IVONA program are very similar to real ones, not bad enough pronunciation, they do not stumble. By the way, the program can be useful for those who are studying a foreign language. Thanks to her, you can listen to the correct pronunciation of certain words, turns.
Supports SAPI5, plus it cooperates well with external applications (for example, Apple Itunes, Skype).
Example (entry of one of my recent articles)
Of the minuses: he reads some unfamiliar words with the wrong stress and intonation. In general, it is not bad enough to listen to, for example, a paragraph from a history book while you go to a lecture / lesson - even more than that!
Balabolka
An excellent program for working with books: reading, cataloging, searching for the desired one, etc. In addition to standard documents that can be read by other programs (TXT-HTML, HTML-TXT, TXT-DOC, DOC-TXT, PDB-TXT, LIT-TXT , FB2-TXT, etc.) ICE Book Reader supports .LIT, .CHM and ePub file formats.
In addition, ICE Book Reader allows not only reading, but also an excellent desktop library:
- allows you to store, process, catalog books (up to 250,000 thousand copies!);
- automatic ordering of your collection;
- quick search for a book from your "dump" (especially important if you have a lot of non-cataloged literature);
- The ICE Book Reader database engine outperforms most programs of this kind.
Also, the program allows you to read texts with a voice.
To do this, go to the program settings and configure two tabs: "Mode" (select voice reading) and "Text-to-speech mode" (select the speech engine itself).
Talker
With Sakrament Talker, you can turn your computer into a “talking” audio book! Sakrament Talker supports RTF and TXT formats, can automatically recognize the file encoding (probably, sometimes you noticed that some programs open a file with “crackers” instead of text, but this is impossible in Sakrament Talker!).
In addition, Sakrament Talker allows you to play large enough files, quickly find certain files. The voiced text can not only be listened to on a computer, but also saved to an mp3 file (which can later be copied to any player or phone and listened to away from the PC).
In general, a pretty good program that supports all popular voice engines.
That's all for today. Despite the fact that today's programs still cannot fully (100% qualitatively) read the text so that a person cannot determine who is reading it: a program or a person ... But I think that someday programs will reach this: the power of computers grow, engines grow in volume (including more and more even the most complex turns of speech) - which means that soon enough the sound from the program will be indistinguishable from ordinary human speech ?!