Chapter 1 Introduction

Visual impaired people (VIP) are a group of persons with vision loss to such adegree as to qualify as an additional support need through a significant limitation ofvisual capability. Data in October 2013 from WHO (World Health Organization)shows that 285 million people are estimated to be visually impaired worldwide, 39million are blind and 246 million have low vision. About 90% of the world’s visuallyimpaired live in developing countries. And 82% of people living with blindness areaged 50 and above[1]. Figure 1-1 shows the number of people blind, with low visionand visually impaired[2]. In the worldwide, cataracts remain the leading cause of blindness in thedeveloping countries. Visually impaired from infectious diseases people havereduced greatly in the last 20 years. 80% of all visual impairment can be avoided orcured. The same survey also showed trends in the causes of visual impairment. Indeveloped countries, the major causes of blindness were degenerative diseases, butin underdeveloped countries, vision loss stemmed from mostly treatable diseases,such as cataracts. In these countries, visual loss tended to accompany impairment in other senses, including hearing or touch. The Table 1-1 indicates the estimatenumber of people visually impaired by age. Independent travel in unknown environment is a daily challenge for visuallyimpaired people. To detect and avoid obstacles most individuals use a white cane,but it fails to protect against head-level obstacles. Some individuals rely on a guidedog, which can help the individual navigate more effectively, but the costs ofacquiring and training a guide dog might be prohibitive. Clearly, these limitationsmotivate the development of assistant technology for increased mobility. Computervision as a substitute for human vision constitutes a powerful tool for developingassistant technologies for the visually impaired. Applications based on computervision enhance these persons’ mobility and orientation as well as object recognition,at the same time guide the VIP how to walk. In this way, two main problems ofautonomous navigation, namely walking and orientation, which are key elementsfor a true independent mobility, are solved[3, 4].The Smart Environment Explorer Stick (SEES) concept[5]was proposed bySMIR team of Laboratoire d’Informatique de Modélisation et d’Optimisation desSystèmes (LIMOS UMR 6158 CNRS) which is a key laboratory in France. SEES isa concept of smart device and dedicates indoor and outdoor environment navigationfor people with visual impairment. The SEES concept contains three sub-systems: aglobal server (iSEE), an embedded local server (SEE-phone) and a smart stick(SEE-stick). This thesis focuses on the design and implement of the SEE-Sticksub-system- object detection. SEE-Stick is a new kind of smart stick and is able toprovide VIP with both the orientation and mobility functions.

..........

Chapter 2 Key methods analysis

2.1 Shape-based method

Appearance-based object detection and recognition methods have been utilizedto numerous real situation, and been demonstrated to have good performance.Considerable effort was spent in the past matching geometric shape models ofobjects to image contours[30]. Although these approaches enjoyed some success, itis clear that finding contours exactly belonging to the shape of an object is a hardproblem. This insight has given rise to an emphasis on local texture descriptors[31],the dominant approach today. These appearance-based descriptors summarize localtexture information in the form of histograms of gradients, shape context, geometricblur[32], and many others. While prominent edges are roughly encoded, exact shapelocation has been replaced by a representation of texture. It is well known, however,that curvature of contours and junctions provide crucial shape information. Thus webelieve it is time to investigate a contour representation alongside appearance-baseddescriptors, examples can be seen in Figure 2-1.However, many of these methods either require good whole-object segmentation,but in the situation with clutter, occlusion, or changes of background, theperformance will be limited; or utilize simple conjunctions of low-level features,which causes crosstalk problems as the number of objects is increased. We areinvestigating an appearance-based object recognition system using a keyed,multi-level context representation that ameliorates many of these problems, and canbe used with complex, curved shapes.

2.2 Color-based method

Color provides powerful information for object recognition. A simple andeffective recognition scheme is to represent and match images on the basis of colorhistograms as proposed by Swain and Ballard. The work makes a significantcontribution in introducing color for object recognition. However, it has thedrawback that when the illumination circumstances are not equal, the objectrecognition accuracy degrades significantly. This method is extended by Funt andFinlayson, based on the retinex theory of Land, to make the method illuminationindependent by indexing on illumination invariant surface descriptors (color ratios)computed from neighboring points. However, it is assumed that neighboring pointshave the same surface normal. Therefore, the derived illumination-invariant surfacedescriptors are negatively affected by rapid changes in surface orientation of theobject (i.e. the geometry of the object). Healey and Slater and Finlayson et al.[33]use illumination-invariant moments of color distributions for object recognition.These methods are sensitive to object occlusion and cluttering as the moments aredefined as an integral property on the object as one. In global methods, in general,occluded parts will disturb recognition. Slater and Healey[34]circumvent thisproblem by computing the color features from small object regions instead of theentire object.From the above observations, the choice which color models to use does notonly depend on their robustness against varying illumination across the scene (e.g. Multiple light sources with different spectral power distributions), but also on theirrobustness against changes in surface orientation of the object, and on theirrobustness against object occlusion and cluttering. Furthermore, the color modelsshould be concise, discriminatory and robust to noise.

CHAPTER 3 SYSTEM REQUIREMENT ANALYSIS ............................31

3.1 THE NEEDS OF VIP...................31

3.2 THE GOAL OF THE SYSTEM .............32

CHAPTER 4 SYSTEM DESIGN....................39

4.1 OBJECT DETECTION ARCHITECTURE DESIGN ................39

4.2 ENVIRONMENT SENSE ................................41

4.3 SHAPE-BASED OBJECT DETECTION............................43

CHAPTER 5 SYSTEM IMPLEMENTATION AND TESTING ...............................69

5.1 THE ENVIRONMENT OF SYSTEM IMPLEMENTATION ..............................69

Chapter 5 System Implementation and Testing

5.1 The environment of system implementation

The hardware of SEE-Stick includes a Raspberry Pi board connected withseveral sensors (a camera, a GPS receiver, an optical encoder, a 3-axis gyroscopeand a magnetometer) an input device (keyboard), two output devices (an earphoneand a vibration motor) and a network device (802.11 Adapter). The hardwareconnections are shown in Figure 5-1. For the object detection module, a singlecamera and the Raspberry Pi board is enough. The Raspberry Pi is a credit card-sized single-board computer developed in theUK by the Raspberry Pi Foundation with the intention of promoting the teaching ofbasic computer science in schools. The Raspberry Pi has a Broadcom BCM2835system on a chip (SoC), which includes an ARM1176JZF-S 700 MHz processor,VideoCore IV GPU, and was originally shipped with 256 megabytes of RAM, laterupgraded (Model B & Model B+) to 512 MB. It does not include a built-in hard diskor solid-state drive, but it uses an SD card for booting and persistent storage, withthe Model B+ using a MicroSD. The Foundation provides Debian and Arch LinuxARM distributions for download.The hardware interfaces of Raspberry Pi include 2 USB 2.0, Ethernet, HDMI,SD card port, 3.5 mm audio jack, and lower-level General Purpose Input /Output(GPIO) interfaces aimed to connect more directly with chips and sub-systemmodules. GPIO is a generic pin on a chip whose behavior (input or output) can becontrolled through software. The Raspberry Pi board has a 26-pin 2.54 mmexpansion header arranged in a 2x13 strip. They provide 8 GPIO pins plus access toI2C, SPI, UART) as well as +3.3 V, +5 V and GND supply lines. In the SEE-Sticksystem, the GPIO pins are configured for sensors (a GPS receiver, a 3-Axisgyroscope, a magnetometer and an optical encoder) connecting to Raspberry Pi (seeFigure 5-2).

OpenCV (Open Source Computer Vision) is a most widely used library ofprogramming functions, mainly aimed at real-time computer vision application. It islicensed under the BSD license, so it’s free for use. OpenCV sopports Linux, MacOS, Windows, Android and IOS. OpenCV focuses on solving the real-timeapplication with computational efficiency. The OpenCV library contains functionsthat span many areas in computer vision, such as product inspection, medicalimaging, security, user interface, camera calibration, stereo vision, and robotics.OpenCV provides a lot of important features: 1) Basic image processing (filtering,edge detection, corner detection, sampling and interpolation, color conversion,morphological operations, histograms, image pyramids); 2) Structural analysis(connected components, contour processing, distance transform, various moments,template matching, Hough transform, polygonal approximation, line fitting, ellipsefitting, Delaunay triangulation); 3) Camera calibration (finding and trackingcalibration patterns, calibration, fundamental matrix estimation, homographestimation, stereo correspondence); 4) Motion analysis (optical flow, motionsegmentation, tracking); 5) Object recognition (Eigen-methods, HMM); 6) BasicGUI (display image/video, keyboard and mouse handling, scroll-bars); 7) Imagelabeling (line, conic, polygon, text drawing).

5.2 Object detection module architecture

According to the techniques utilized, the object detection moduleimplementation overview can be shown in Figure 5-4. As shown in the Figure 5-4,the object detection module gets the location information from the SEE-Sticknavigation module. The information includes the accurate position and landmarkobject image, and environment image. The environment sense module gets theimage at a certain location and compare the histograms, after these processes sendfeedbacks to the SEE-Stick interface layer to give notice to the VIP. For the objectdetection, retrieve the object image, and segment the object, then extract thefeatures to the pre-trained SVM classifier to identify the object. After the object isdetected and recognized, the object’s location information stored in the database canbe compared with location information gathered by all kinds of sensors installed inthe SEE-Stick to calibrate the location. And this error is send to the SEE-Stickinterface layer to calibrate. To calculate the color histogram, first the color space should be divided intoseveral small color space, every interval becomes a bin of histogram. Then cal culatethe quantity of color in every interval can get the color histogram. OpenCVprovides basic data structure and functions for the histogram, including calculation,normalization, compare and so on. In the OpenCV, histogram structure are definedas CvHistogram (see Figure 5-5).

..........

Conclusion

This thesis aimed at designing and implementing a real-time, low cost andenergy efficient smart stick static important objects detection module to enable VIPto travel in indoor environment safely and independently. It firstly reviewed presentsmart sticks, however, none of them fulfilled both mobility and orientationrequirements. To enable low cost navigation and improve the navigation accuracy,this paper suggested fusing several sensors including gyroscope, wheel encoder andmagnetic compass, ultrasound and camera. After analyzing the indoor environmentwhere VIP are moving, this research suggested three functions for the objectdetection module: environment detection, static important landmarks detection anddata fusion function. Considering the scalability, different methods were proposed,including shape-based method, color based method, feature-based method andmachine learning method. And the advantages and disadvantages of all these methodwere discussed, at last, a new method combine all these methods were proposed. Thelandmarks detection result can be fused other sensors. With the angular rates andvelocities obtained from gyroscope and wheel encoder respectively were fused byDead Reckoning method to predict VIP’s positions, headings and velocities. Thepredicted results were calibrated by Kalman Filter using GPS measurements andcamera detection results. The navigation engine used the calibrated results to analyzeVIP’s positions and behaviors to give out appropriate instructions and guide VIP todestinations. The system prototype demonstrated promising results in indoorenvironment. The results validated the object detection method and multiple sensordata fusion method provided positioning solution accuracy and reliability.As an innovative smart stick technique, The SEE-Stick object detection modulecan be further enhanced. Several future works are suggested. Firstly, for detecting allkinds of doors or emergent signs, the generic algorithms should be considered;Secondly, as the indoor environment is quite complicated, more complicated sensorslike 3D camera may need to be introduced to enhance the accuracy in navigation andcalibration in position; Thirdly, the histogram match’s accuracy can be improved.

........

参考文献（略）