%====================================================================== %---------------------------------------------------------------------- % XX X % X % XX XXX XXX XXX XXX X XXXX % X X X X X X X X X X X % X XXXXX XXXXX XXXXX X X XXX % X X X X X X X X % XXX XXX XXX XXX XXX X XXXX %---------------------------------------------------------------------- % SPECIFICATION FOR COMMON IEEE STYLES %---------------------------------------------------------------------- % Gregory L. Plett, Istv\'{a}n Koll\'{a}r. %====================================================================== \documentclass[% %draft, %submission, %compressed, final, % %technote, %internal, %submitted, %inpress, reprint, % %titlepage, notitlepage, %anonymous, narroweqnarray, inline, twoside, invited, ]{ieee} \newcommand{\latexiie}{\LaTeX2{\Large$_\varepsilon$}} %\usepackage{ieeetsp} % if you want the "trans. sig. pro." style %\usepackage{ieeetc} % if you want the "trans. comp." style %\usepackage{ieeeimtc} % if you want the IMTC conference style % Use the `endfloat' package to move figures and tables to the end % of the paper. Useful for `submission' mode. %\usepackage {endfloat} % Use the `times' package to use Helvetica and Times-Roman fonts % instead of the standard Computer Modern fonts. Useful for the % IEEE Computer Society transactions. %\usepackage{times} % (Note: If you have the commercial package `mathtime,' (from % y&y (http://www.yandy.com), it is much better, but the `times' % package works too). So, if you have it... %\usepackage {mathtime} % for any plug-in code... insert it here. For example, the CDC style... %\usepackage{ieeecdc} \usepackage{graphicx} \begin{document} %---------------------------------------------------------------------- % Title Information, Abstract and Keywords %---------------------------------------------------------------------- \title[SHARIKA-Malayalam Speech Recognition System]{% SHARIKA \\ Malayalam Speech Recognition System} % format author this way for journal articles. % MAKE SURE THERE ARE NO SPACES BEFORE A \member OR \authorinfo % COMMAND (this also means `don't break the line before these % commands). %\author[Shyam]{Shyam.K},\authorinfo{Shyam is with the Department of Electronics And Communication % Engineering, MES College of Engineering,Kuttipuram % Phone: $+$919946787953\,0494-2667303, e-mail: shyam.karanattu@gmail.com} %} % format author this way for conference proceedings \author[SHYAM]{% Shyam.K\authorinfo{% Department of Electronics and Communication Engineering,\\ MES College of Engineering,Kuttipuram,pin 679573.\\ Phone:$+$919946787953,\,0494-2667303,email:shyam.karanattu@gmail.com}%s } %\and{}and% %\and{}Istv\'{a}n Koll\'{a}r\member{Fellow}\authorinfo{% %Department of Measurement and Instrument Engineering,\\ %Technical University of Budapest, 1521 Budapest, Hungary.\\ %Phone: $+$\,36\,1\,463-1774, fax: +\,36\,1\,463-4112, %email: kollar@mmt.bme.hu} %} %\journal{ICIST Trans.\ on Instrum.\ Meas.} %\titletext{, VOL.\ 46, NO.\ 6, DECEMBER\ 1997} %\ieeecopyright{0018--9456/97\$10.00 \copyright\ 1997 IEEE} %\lognumber{xxxxxxx} %\pubitemident{S 0018--9456(97)09426--6} %\loginfo{Manuscript received September 27, 1997.} %\firstpage{1217} %\confplacedate{Ottawa, Canada, May 19--21, 1997} \maketitle \begin{abstract} Sharika - an Automated Speech Recognition(ASR) system is an initial initiative in malayalam speech recognition.ASR in its wider perspective is still a dream project,in malayalam.Though the speech corpus required for building the models are ready built,we cannot make use of those corpus as those are given under non-sharable licence.Since ASR in malayalam in its true sense is a herculean time consuming project,here our endeavour is to make a pilot project limiting the word count by confining the ASR to a particular task namely voice control for desktop commands. \end{abstract} \begin{keywords} Automatic Speech Recognition,Sharika,CMUSphinx,Free Software,Malayalam. \end{keywords} %---------------------------------------------------------------------- % SECTION I: Introduction %---------------------------------------------------------------------- \section{Introduction} \PARstart Among the most ancient human communication systems, verbal form of communication is the most scientific mode. When we go deep in to the human history,we can find that vocal communication was effective far before written methods. In this most modern age also one cannot underestimate the effectiveness of speech as a mode of communication. If our computer can occupy the place of a stenographer, read news according to our choice,can teach students in their own mother tongue,it will be a major break through.We leave the practical applications of speech recognition to the imagination of the readers as it is very vast.This effort becomes invaluable for those with physical inabilities.Our endeavour here is to minimize the digital divide by making computer applications more user friendly-just by spelling it! that too in our mother tongue. %---------------------------------------------------------------------- % SECTION II: The Document Life-Cycle %---------------------------------------------------------------------- \section{MALAYALAM} Malayalam belongs to the Dravidian family of languages and is one of the four major languages of this family with a rich literary tradtion. Malayalam is the official language of the state of kerala ,India.Its spoken by around 37million people. Indian language at large and malayalam in particular are very phonetic in nature.The structure of alphabets in malayalam is arranged based on the method of sound production.As we pass through the consonant list ,we can see that its so arranged that the noice source of consonant sound, moves from inner mouth to lips. \begin{figure} \includegraphics[scale= 0.5width = 60mm]{formant-graph.png} \caption{Shows the graph of first formant frequency versus second formant frequency for malayalam vowels for various speakers} \end{figure} % Screenshot-1.png: 1024x768 pixel, 72dpi, 36.12x27.09 cm, bb=0 0 1024 768 %\end{center} %\includegraphics[width=60mm]{praat.eps} According to KeralaPaniniyam the alphabets of malayalam consists of 16vowels and 37 consonants,though this has been to slight modifications through ages.The list of vowels is made of 4 short vowels and 3 long vowels in the division of samanakhshara, 2 short vowels and 4 long vowels in the division of sandhyakshara and two more as അം[aum] and അ:[ah].The figure below shows an illustration on how the formant frequencies vary for some of the vowels. The division of consonant sounds are even more interesting. Consonant sounds are produced by partial or complete closure of upper vocal tract.So taking the famous analogy of method of speech production with electrical networks,we have the noice sources changing their place along the vocal tract,as the consonant changes.Interestingly in malayalam, they are divided on the same basis of the place where the flow of air is affected. %---------------------------------------------------------------------- % SECTION III: Specifications %---------------------------------------------------------------------- \section{Speech Recognition} This project uses the Sphinx engines developed by Carnegie Mellon University ( CMU). SPHINX is one of the best and most versatile recognition systems in the world today.Sphinx is having a BSD derived license so people can share the software and use it as they wish.But propreitary software authors can also use sphinx to make propreitary software due to the famous liberal attitude of BSD license. Sphinx has many versions which meets different environments in which the system is employed.They are sphinx2,PocketSphinx sphinx3 and Sphinx4.PocketSphinx is CMUSphinx's most accurate ASR system while its not as accurate as Sphinx3 or Sphinx4.Sphinx3 is using continuous HMM while PocketSphinx and Sphinx2 are using semi-continuous HMMs.Sphinx4 is written entirely in JAVA-programming language. Sphinx train is the acoustical model trainer software in CMUSphinx. It is a statistical acoustic model trainer using Hidden Markov Models (HMM). Due to inavailability of a speech corpora, the project is designed to have a wrapper around sphinxtrain so that people can easily train the acoustic models for sharika and use those models to do the desktop control.So A new user first trains his model for sharika and then uses the system.This will greatly improve the accuracy of the model as a near-perfect user independent model itself requires long-term project of collection of speech corpora and tuning the models to perfection. The desktop control part is developed over the GNOME libraries so that the GNOME users can control their window navigation and general desktop control using this system.Initially I developed a simple 8 command system which is now enhanced to above 50 commands. The sphinx decoder requires two things to perform decoding. \begin{enumerate} \item Acoustic model and \item Language Model \end{enumerate} \subsection{Building The Acoustical Model} Acoustic model is developed using sphinxtrain package.Sphinx train has the training algorithms implemented so that we can configure the system and give the required data i.e. both speech and other configuration files such as dictionary, phonelist and transcripts to build the models in the way we want.The task of defining the phonemes is much simple,thanks to the phonetic nature of malayalam language. Majority of the letters themselves form phonemes.We can remember the fact that almost all written malayalam words unlike english words can only be read in one way which shows how phonetic they are.As the project requires users to train their own models,the speech database made to train the models uses only my sound.Phone list,dictionary file and transcripts specific to this project are made.Users are expected to read out the transcripts as they do the training as per the trainer program. As the user completes audio recording ,the recorded sounds are processed to get the feature vectors extracted.Mel Frequency Cepstral Coefficients(MFCC) are by far the favorite feature vector for ASR systems.Here we are trying to model the response of ear and have many narrow filter banks below 1000Hz and gradually wider bands above;using mel scale to define the width.Thus a frequency wrapped version of cepstrum is used.Feature vector sequence is now used to train acoustic models. Continuous HMMs are made first in the training session which is later converted to Sphinx2 models so that they can be decoded using PocketSphinx %---------------------------------------------------------------------- \subsection{Building The Language Model} Being a project with small word count, I used Finite State Grammer as the language model and an fsg file is handwritten for the purpose.A screenshot of which is given in the illustration below. %\begin{figure} %\includegraphics[scale=0.25]{fsg.png} %\caption{Shows the graph of first formant frequency versus second formant frequency for malayalam vowels for various speakers} %\end{figure} These models are given to sphinx decoder to do the decoding.Sphinx decoder APIs are used along with the libraries related to desktop environment to perform live decoding. The trained models being user dependent,also as the no: of words are low, is having very high performance and is working well even for male voices similar to that of mine.As the project is using a user dependent model the accuracy measurements are giving above 90 percent correctness and low word error rates. \section{Conclusion And Future Work} ASR systems are very complex and is a very time consuming project.Members of Swathanthra Malayalam Computing are now doing the ground works of building a Free Text Corpora for malayalam which will be in a sharable license.Building a speech corpora is even bigger activity and requiring more time and effort.But once built,those models can be adapted and used for enormous number of applications.As already stated, Speech recognition programs are having varities of applications and will become a revolutionary breakthrough when we consider the low percentage of literacy of our country.A speech-to-speech application which interactes with the user by accepting speech and giving out speech could be one of the best use of speech recognition systems,where the user gets the maximum convinence. With world-class Free Software ASR systems like Sphinx which is built and through contribution from people all over the world,the task of building speech recognition programs are becoming more and more simpler.Still the bigger task remain for us - the making of a user independent acoustical and language model for malayalam, \section{Acknowledgement} I would like to acknowledge the continuous support and help from my parents and friends and the members of SMC. \begin{thebibliography}{1} \bibitem{Tutorialn} L Rabiner \newblock {\em A Tutorial on Hidden Markov models and Selected Applications in Speech Recognition , Proc. Of IEEE, Vol. 77 No. 2, 1989.} \bibitem{GOLD} Ben Gold and Nelson Morgan \newblock { \em Speech And Audio Signal processing,processing and perception of speech and music} \bibitem{Digital} L R Rabiner,R W Schafer \newblock { \em Digital processing of speech signals} \bibitem{Quatieri} Thomas F Quatieri \newblock { \em Speech Signal processing Principles and Practice} \bibitem{Keralapaniyiam} \newblock { \em keralapaniniyam and common malayalam grammar books for the detailed reference of malayalam language.} \bibitem{CMUSPHINX} \begin{verbatim} http://cmusphinx.org/ \end{verbatim} \newblock {\em above is the central link to the complete Sphinx project at CMU} \bibitem{Subwiki} \begin{verbatim} http://sphinx.subwiki.com/ \end{verbatim} \newblock {\em above is a newly created wiki.} \bibitem{Fundamentals} Lawrence Rabiner and Biing-Hwang Juang \newblock {\em Fundamentals of Speech Recognition} \end{thebibliography} \end{document}