How to check if a picture is an image or a floor plan by using Javascript4 min read

Hi there! Oh, maaaan… This Coronavirus situation was pretty difficult. I hope you all managed to overcome it without any serious consequences. As you might have guessed, it did impact my writings on my blog and youtube video recordings as well.

The home office proved to be much more exhausting than it sounds at first. My productivity did increase but it was hard to separate work from leisure. But that’s going to be a blog post for itself.

Let’s get going with some new cool content. 

The challenge

Recently, I had the opportunity to work on some awesome reactjs project. And I stumbled upon a feature request to determine if a picture is an image or a floor plan by using Javascript. And without using any Google, Microsoft, machine learning, AI whatsoever hyper-cool trending stuff.

What does that mean? 

Well, it means that you need to upload some photos and determine if it’s a proper photo taken by a camera or if it’s a floor plan (schema). The very tricky part of this feature was that the floor plan can be in color, not just black and white. Which would significantly make my life easier of course? But hey, we love challenges! Especially this kind of Javascript challenges, don’t we!? 🙂

via GIPHY

The search

After googling around for a while I couldn’t find a proper copy/paste solution. Even more, I had a problem by only finding how to use google, to search what I needed. And after a while, I ended up on some forum posts which gave me an idea of how to solve this issue.

via GIPHY

The whole idea is to load an image and check each pixel of an image then based on that make some calculations regarding color size and saturation.

The solution (theory)

To determine if one photo is a floor plan or not:

1. Check photo saturation and if it’s lower then XYZ it’s a floor plan
2. Check the number of pixel colors. If it’s below certain color number then it’s a floor plan 

Okay, number of colors makes sense but what the heck is saturation?
If you google it, you will find a definition which says:

“Saturation refers to the intensity of a color. The higher the saturation of a color, the more vivid it is. The lower the saturation of a color, the closer it is to gray. Lowering the saturation of a photo can have a “muting” or calming effect while increasing it can increase the feel of the vividness of the scene.”

So black and white colors have saturation 0 (zero) or almost 0 (zero) while photos are more saturated. So, camera taken photos is more saturated than the floor plan images (although) this might not always be the case.
The higher the resolution the image is more saturated. That’s why this solution isn’t 100% bulletproof. But does cover more than 98% of cases which was definitely more than enough.

via GIPHY

Pseudocode

Okay, let’s get more into pseudocode which says the following:

• Obviously, load the image into pixel array
• You should implement the function which loops through each pixel of an image, and then you need to find the average saturation number
• If that number is below the constant number we set by ourselves (which proved to be around 0.03, but you can tune this a little bit more) we can assume it’s a floor plan.
• To make our solution more rigid we will also implement a function that shall check the number of colors. 

Oh, by the way, I will only show you the snippets of code. If You want to challenge yourself, feel free to create a custom library based on this.

I will provide you in the end a more pseudocode steps to ease you the process.

The (partial) solution

getSaturation({ pixelArray, pixelCount }).then((saturationValue) => {
            if(saturationValue < 0.03){
              getPixelColorsSize({ pixelArray, pixelCount, imgData }).then(colorSize => {
                if (colorSize < 1000){
                  accept(true)
                } else { 
                  accept(false);
                }
              });
            } else {
              accept(false);
            }
          });


const getSaturation = ({ pixelArray, pixelCount }) => {
  return new Promise((resolve, reject) => {

   const sum = pixelArray.reduce((total, pixel) => total + calculateSaturation(pixel));

    const checkValue = sum / pixelCount;

    resolve(checkValue);
  });
};


const calculateSaturation = (pixel) => {
  let r = pixel[0];
  let g = pixel[1];
  let b = pixel[2];

  // Make r, g, and b fractions of 1
  r /= 255;
  g /= 255;
  b /= 255;

  // Find greatest and smallest channel values
  const cmin = Math.min(r, g, b);
  const cmax = Math.max(r, g, b);
  const delta = cmax - cmin;
  let s = 0;
  let l = 0;

  // Calculate lightness
  l = (cmax + cmin) / 2;

  // Calculate saturation
  s = delta === 0 ? 0 : delta / (1 - Math.abs(2 * l - 1));

  return (s);
};

Conclusion

I hope this article helped you a lot if you stumbled upon this kind of feature request. I surely know, if I had it it would be a breeze to implement it.

Anyways, thanks for reading and let me know if You managed to create a custom library based on this?

Cheers

The challenge (homework)

More pseudocode:

1. Load image
2. Get pixel array (hint: use get-pixels library)
3. Use above explained saturation code
4. Loop through pixel array and add each pixel r-g-b value to a collection to check color size