Controlling railway crime with passenger counting and facial recognition technology

The Passenger Rail Agency of South Africa (PRASA) is mandated by government to provide passenger rail services across South Africa. In recent history, it has been brought under a huge amount of pressure, due to sabotage and arson. Since 2016, over 140 carriages have been destroyed by arson, with only a small number of vandals being prosecuted. To make matters worse, and as a result, it is common practice for carriages to be overloaded with commuters using rail to travel to and from work. This causes damage to train carriages’ structures and suspensions. There is also strong evidence of general vandalism and theft.

PRASA has funded a Research Chair in the Department of Industrial Engineering at the University of Stellenbosch. The Chair, in collaboration with the Department of Mechanical and Mechatronic Engineering, was initially tasked with developing a system which would count the number of passengers entering and leaving a carriage, so that railway staff could control numbers and check tickets. The data collected could be used by the planners at PRASA to optimise train schedules and minimise overloading caused by too few trains. With the increase in crime on rail lines, PRASA requested that the project be expanded to include facial recognition services to be run in conjunction with the South African Police Service (SAPS) to identify, and then apprehend, known criminals.

The project has been running for the past four years using final-year engineering students and has achieved notable success as a pilot project. Both passenger counting and facial recognition have proven to be successful under controlled conditions and are due to be tested in live field trials during 2020.

The initial project concept used door-facing cameras to count faces and, from this, the idea of including facial recognition developed. Faces would be recorded at the ticket office and then checked on entry to the carriage, thereby providing an opportunity to identify freeloaders. The sheer size of the data that needed to be stored soon showed that this concept was impractical. However, it was suggested that facial recognition could be used to compare passengers with a stored database of people wanted by law enforcement. A warning would be transmitted to a central control room and the authorities would be waiting at the next station to arrest the wanted person.

Because of the environment in which the project was developed, the budget was minimal, and the use of freeware and shareware was essential¹.

Passenger counting

The use of cameras was fundamental from the beginning. Initial use was made of single lens cameras, but situating the cameras facing the doors proved impractical because of the huge range of light intensity the cameras would have to deal with. It soon became apparent that vertical stereo cameras placed above the door would give the best solution. The problem then became one of identifying individual heads as they entered and exited the doors, and so the development of a suitable algorithm was needed to make the required count.

Depth Map (Neser, 2019)

After much research and experimentation, the Depth Map system, using the Waterfill algorithm developed by Zhang and others², was used. The Depth Map created from a 3D camera, a landscape with peaks and troughs where simulated rainfall could accumulate, where the algorithm identifies the peaks as representative of a passenger’s head. Thus, it can count multiple heads of different heights simultaneously. The software was further made capable of identifying in which direction the heads were moving. This allowed counts to be made monitoring movements in both directions, resulting in the net flow of passengers. The information was stored locally for onward transmission to a central website.

The software was developed, originally, using MatLab. It was later converted and improved using Python. This was done following a policy decision to use as much freeware as possible. Furthermore, Python could process the data more efficiently and integrate easily with the facial recognition code. The system was tested under poor lighting conditions and correctly counted with an accuracy of 70 per cent.

The process steps were:

• Initialise the device
• Generate the Depth Map stream
• Process Depth Maps
• Track objects
• Increase or decrease the passenger count
• Store the count locally on the train
• When suitable, transmit the data to a central database.

Facial recognition

Facial Recognition (Neser, 2019)

As a result of the first concept, which proposed the use of camera facing the doors, a call was made to add facial recognition. It was quickly apparent that the amount of data storage that would be required would be excessive if every face was to be captured. This would be further complicated by image processing if it was intended to compare facial images with a stored database of sought people. A facial processing algorithm was developed which reduced the size of the data that needed to be compared with similar process data supplied by SAPS. The captured face was reduced to essential biometric data in both cases. The deep learning algorithm software developed was able to identify faces even when they were turned sideways to the camera.

The process steps were:

• Initialise the device
• Generate the RGB stream
• Process RGB frames
• Detect faces
• Identify faces
• Store recognition results locally on the train
• When suitable, transmit the data to a central database.

Processing

It was found that trying to compute and correlate the data captured resulted in delay times, which affected the accuracy of the passenger count. The facial recognition facet required too much of the available resources using a single processor. As a result, the passenger count system was delegated to a stand-alone Raspberry Pi, which allowed the systems to have an accuracy of 90 per cent for facial recognition and more than 75 per cent passenger counting. The other handicap was that the project was severely financially restricted, and the results would be significantly improved with better cameras. For instance, the camera which counted passengers had a minimum focal length of 400mm, while the door height was 2.1m, meaning anyone over 1.7m tall would not be counted.

Ongoing project

The project is not yet complete and will be worked on during 2020 to improve and finalise the system. It is expected that more suitable cameras will be sourced, which should increase the accuracy of passenger counting. The system must still be tested under field conditions.

References:

1. Zhang, X., Yan, J., Feng, S., Lei, Z., Yi, D., & Li, S.Z. 2012. Water filling: Unsupervised people counting via vertical Kinect sensor, 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS). 18-21 September, Beijing, China.
2. Neser, J.E.  2019. Development of a passenger counting system incorporating facial recognition for the passenger rail environment, Department of Mechanical and Mechatronic Engineering, University of Stellenbosch.