Current Implementation
In getting to our current state we faced several hurdles and at times had to completely change our approach. To read about those early challenges, see the "Failed Approaches" section below. Similarly, our current approach also presented several challenges. Listed below are the top 4 challenges our current implementation presented. These are listed in order of their appearance during the project.
- Which edges to use for identifying the license plate.
- This early challenge was difficult because there are a number of edges that show up around the license plate--the bumper, window, car siding panels, and the letters on the plate themselves. Picking which edge to target from this list was difficult. It wasn't until after careful examination of many different images that we decided to target the vertical side edges of the plate. These edges were the only ones that seemed to be consistently distinguishable from each image.
- How to filter out the appropriate edges
- Solving this challenge was the key to our whole approach. The intelligence of our algorithm lies in its filtering, and creating consistently accurate filters proved very difficult. There are few differences between the vertical edges on the plate and those seen on some of the car back windows. Thankfully, windows are often curved, and so eliminating edges that sat at a partial angle in the image removed this problem. However, this success came at a cost, it also eliminated the ability to handle any image taken at an angle. Any edge that is not straight up and down will be eliminated via this filter. This tradeoff was one made to ensure that this approach would function in the most basic sense.
- How to determine the final pair of edges to use
- This was difficult because even after the many rounds of filtering, multiple vertical edges still remained. Often, vertical edges from the license plate letters would present themselves as the best candidates, which would generate a cropped image too small for deciphering. This problem was eliminated by using the 2:1 aspect ratio that many license plates adhere to. This fact places the two desired edges at a known distance apart from each other. From here, the remaining edges were filtered until a pair was found such that they were close to twice their own height apart. This eliminated the problem with the letter edges. However, it also prevents the algorithm from being successful on certain international plates. Some plates, like those in Great Britain, do not have a 2:1 aspect ratio. This parameter can be tweaked within the code, however its presence prevents universal success.
- How to read the text off of the plate
- This problem, solved in many different ways online, was tough to implement for this use case. Matlab's built in text reader, optical character recognition (ocr), was the first logical choice. However, it found almost no success given the input image of just a license plate. Often, it pulled characters from parts of the plate that were irrelevant, and failed to read even the key letters and numbers. After many attempts, including filtering the input fed to the function, the lack of success became frustrating. As a second approach, we attempted to compare the extracted letters to an alphanumeric database. Simple, and fairly effective, this was the one chosen for current implementation. However, we acknowledge that it could be better. Many of our current failures come from the weaknesses of this database approach. A number of corrections had to be made, such as turning certain 'O's to '0's or 'Z's to '2's if they are next to other numbers on the plate. In the future, a better form of text recognition should be used, for instance one based on deep learning. A small improvement here could make this program much more effective.
Failed Approaches
Over the course of this project many attempts were made at accomplishing the task at hand. The majority of these approaches failed. Listed below are a few of our initial attempts and the reasons we chose what we did.
Failed Approach #1: Using What is Already Out There
Prior to beginning this project we looked around for any available code that had been developed in an attempt to read a license plate. If a widely available, successful program already existed, then we would have to choose a different project. We found two individually developed Matlab programs that claimed to accomplish what we wanted to do.
They performed well in certain circumstances, but were largely ineffective as a whole. They were not consistent and did not do much more than basic image filtering. They avoided the challenge of image segmentation and attempted to just pull out the text from the original image. For a while, we attempted to modify these two approaches with what we knew about image processing. However, our attempts were unsuccessful given we were applying them to the entire image. These approaches did not focus in on the license plate, and were not as good because of that. We gleaned a lot about filtering out noise from an image, but ultimately we abandoned these approaches. To test these programs yourself, see:
- Vehicle Number Plate Recognition
- License Plate Recognition
Failed Approach #2: Image Segmentation Via K-Means
After confirming that we must indeed, cut out the license plate from the image, we centered on a new approach: K-Means clustering. This method, from class, we hoped would help to find the license plate from within the image and allow us to crop it out. Upon several rounds of testing, we were able to get the license plate removed from a few images. However, the program was very inconsistent, and more often than not it would fail in removing the license plate. Images are different, and it seemed to require that we adjust the number of clusters for different images. It also depended heavily on color, and if parts of the license plate had different reflections, they would not always be a part of the same cluster. For these reasons, we abandoned K-Means clustering as our method for removing the license plate.
