Image processing can get pretty compute power consuming…

I wrote earlier a post on video surveillance using Python (see this post) – it works good and the neat thing is the Flask application that allows you to stream your video on a website. However I was not impressed by the speed of the video – it does not get smooth enough on a Raspberry Pi 3 – especially not if you start doing some more image processing… So I was wondering if things can get sped up using C++.
Here is an example with my cat detected:

Larger C++ projects can get hard to oversee. Fortunately you can take advantage of class structures and modularity of your library to have everything better organized.
Another challenge I always face is how to compile the entire thing in C++, especially when you have multiple modules. I found that in such a case cmake is a very helpful tool. The final result is an executable file.

Finally I want to show how you can set it up that the videos that are captured are automatically loaded on your cloud Dropbox folder, so that you can access them even with your smart-phone on a trip.

Motion Visualization with C++

There are several ways to get video streaming with C/C++, such as importing and using on Linux the native functions ioctl and video-for-Linux v4l (see e.g. the great post of JWH Smith).

OpenCV has some very useful routines to make the image/video capture in C++ pretty simple. This comes handy if you are also using openCV to do image processing with the captured images.

Required for this Project

This post builds on the openCV installation detailed in Webcam Video Access with OpenCV on Raspberry Pi. You will need OpenCV installed on your Raspberry Pi to be able to run the following code.
I was using as operating system (OS) the standard Raspberry Pi Raspbian environment. But cmake should make the entire code easy transportable to other systems.
The entire code can be downloaded from GitHub by typing at the command prompt:

The hardware I was using was consisting of:

  1. A Raspberry Pi 3 — This allowed a framerate of about 20fps (even though the code shows 30fps…) — with a 64GB SD card (such as this one). However 16GB would mostly be large enough.
  2. A Logitech webcam (such as this one).

…and the other usual periphery you need for the Raspberry Pi such as monitor (if you are not using remote login with SSH), keyboard and mouse.

The Structure of the C++ Library

In programming languages such as C/C++ you should always have a clear plan of the structure you need and the tasks your program should perform. In this flowchart for main.cpp you can see a flowchart that basically describes everything our main program does.

The file and folder structure for the C++ code might seem slightly complicated on first sight, but the modular build-up helps you stay organized once the code gets larger.

The files and folders for the here presented C++ routine are:

./include and ./detections are two folders. Here ‘time_stamped_video_files.avi’ are the output video files generated by the code when motion is detected. So at start the folder ./detections is actually empty (or has the README file from the GitHub repository).
The files in some more detail:

  • main.cpp: The main routine that creates instances of the class library.
  • include/camLib.hpp: The definition of the applied classes.
  • include/camLib.cpp: The member functions of the applied classes.
  • CMakeLists.txt: cmake library that enables simple compilation with openCV bindings.

The following chapters show the implementation and brief explanations of what is going on. Finally I will give you a description how the captured motion videos can be automatically pushed onto your Dropbox ‘cloud’ folder, which you can access from almost anywhere in the world where you are.

The Main Function

The main function main.cpp has the purpose to activate the camera and then in an endless loop to find out if anything moved in the field-of-view (FoV) of the camera. This all is achieved by calling member functions in ./include/camLib.cpp, which itself calls e.g. openCV functions.

A  main goal was to avoid writing video files on the SD card just to delete them once no motion was detected. This would reduce the life of the SD card. Better is to collect video frames in the memory (RAM) and only write the frames onto the SD card if motion was detected. If a sequence contained no motion the RAM is easy to erase.

Here a listing of main.cpp:

An instance of the Camera class is defined on line 6, called cam1.

On line 28 a memory stack named frameStack is defined which will collect the video images in RAM memory. Only if motion is detected will this stack be saved on SD card as a .avi video.

In line 37 an endless loop starts (ending on line 108). This endless loop is only terminated by pressing the Esc key, which is given in the code on line 90 to 94. Inside of this loop following steps are performed:

  1. Capture a frame (line 39)
  2. Collect a sequence of frames counting in frameCounter from 1 to frameBuffer (=50 on line 8).
    • If motion was detected (with the help of the function detectMotion()):
      • Was the motion observed in a continuous sequence of lengthThreshold (=5 on line 9) frames?
        • if yes then set the flag saveVideo to 1 (on line 51)
        • also set the frame counter frameCounter back to 0. This will make the captured sequence longer depending on for how long motion was detected.
      • If the continuous frame sequence was not long enough then reset the counter for the frame sequence frameSequence (on line 56).
  3. If finally frameCounter is larger than the value for framebuffer and additionally motion was detected in the sequence that was saved on the stack (indicated with saveVideo=1), then:
    • Reset flag and counters
    • Generate a file name with the date and time stamp
    • Open a video file with the generated file name (on line 73)
    • On lines 75 to 78 then are the frames from the stack taken on-by-one and saved in a .avi video file
  4. If frameCounter is larger than the value for framebuffer but no motion was detected (indicated with saveVideo=0), then:
    • Set all counters and the flag back to zero.

As mentioned earlier the flowchart for main.cpp can help get a better overview of the flow if the description here is not sufficient.

The Header File

The header file (here with the name camLib.hpp) defines the implemented classed and is basically a listing of all element functions and variables used in this class.
It also contains a listing of all required include files which are used in all other parts of the program.

The functions declared in lines 16 and 17 are the constructor and the destructor of this class, respectively. This is where the camera is activated and deactivated. The constructor prepares many things that are needed to start with the motion detection.
The other function in the public block of camLib.hpp are more in detail described below.

Important here is the definition of the VideoWriter object writer which is used below to write frames captured by the camera on the SD card.

The Member-Functions of the Camera Class

The functions declared in the header file are defined (implemented) in the member function file camLib.cpp. Here is the actual formulation of the functions.

I will keep the documentation regarding the above file camLib.cpp only to the most essential points. More documentation is given a comments in the code.

camLib.cpp starts with the instantiation of the camera in line 5 as the global variable cap.

The class constructor in lines 9 to 44, which gets into action only once at the definition of a class instance, initializes and prepares everything:

  • Line 11 starts the camera.
  • Lines 22 to 27 capture 3 frames, which are used as initialization of the motion detection.
  • Line 43 defines the ‘codec’, which is required to write later the .avi video file on the SD card.

In lines 47 to 50 is the destructor of the class, which gets into action only once at the termination of the program and makes sure that all processes end properly.

The function captureVideo() in lines 54 to 72 captures a frame and calls the function diffImg() with the current and the two last images (called prev_frame, current_frame and next_frame).

The function diffImg() in lines 75 to 83 calculates (prev_frame – next_frame) and (next_frame – current_frame). These simple calculations leave everywhere ideally a 0 in the resulting two frames where nothing moved. But even if nothing moved unavoidable noise will result in not zero values even if nothing moved. An “AND” operation between the two resulting frames helps reduce false non-zero values.

The function detectMotion() in the lines 87 to 123. The openCV function meanStdDev() called in line 89 grabs the calculated motion image from line 69 and determines mean values and standard deviation in this image. With this we have a base-level that will help find ‘real’ motion.

This standard deviation is used as a ‘benchmark value’ in line 92, comparing it with a user-chosen limit-value max_deviation to decide if the camera detected motion. Motion is detected in the motion image from line 69 where the values are close or equal to 255. In lines 105 to 117 those areas are framed in by a yellow box to highlight where exactly the motion in the image is taking place.

The function openVideoFile() in lines 127 to 135 opens a new file.

The function saveVideo() in lines 139 to 148 uses the VideoWriter object writer to write frames on the SD card.

Linking and Compiling with CMAKE

Traditionally to compile and to link the modules or libraries of a C/C++ project together to get a functioning executable file, a Makefile is used. Writing a Makefile can get tricky, if a project is a little more complex.
CMAKE is a great tool to ease this pain somewhat. It is basically a tool that helps generate the more complicated Makefile, which then – using the command $ make links and compiles all libraries to one executable code.
The cmake I am using here to link and compile the above files is as follows and needs to be saved as CMakeLists.txt :

Detailled information on cmake can be found on the site.
The first two lines in this file define this file as a cmake script and specify the project name. The name of the final executable is defined in lines 14 and 15 as displayMotion.

Compile and Run

Finally we get to compile the whole thing and to run it…

As mentioned you need to first run cmake, which is done by typing and executing:

(Don’t forget the ‘.’)
This generates a Makefile. Now comes the actual compilation with running

Now you can run the executable by typing:

A window should pop up on your screen showing a life stream fro your camera. When something moves in this video it should be framed by a yellow box.

Displaying the Saved Video

The video saved as .avi can be viewed on the Raspberry Pi with omxplayer.
You might have to install this first by typing:

To run the video simply type at the prompt:

Loading Videos onto DropBox

Saving the videos with detected motion on your SD card is already a good surveillance system. Better would be though if the videos are not only saved locally on your Raspberry Pi, but also saved in e.g. your DropBox folder, from where you could check for things happening from anywhere if you have internet access.
So first you need to have a DropBox account. If you don’t have that already, then please just follow this tutorial.
To setup your DropBox uploader on Raspberry Pi you can just follow this great tutorial. However the script from Andrea Fabrizi has become easier to use:
After logging into your DropBox account at, you will find further down on the page Generate access token with a button “Generate” below. After pressing that button you will get a really long random number/letter word. This word needs to be copied at the “Access token:” request when you run Andrea Fabrizi’s script as shown in below image:

Let’s say then that the DropBox folder you created is called “diningroom”. If you change now on Raspberry Pi into the folder where your motion videos are saved, you can upload all of them on your DropBox folder with following command:

Certainly you want this upload to be done automatically. This can be done using Linux’s crontab. Open up crontab with the following:

In this editor add the following lines:

Line 2 uploads every 15 minutes saved videos of the Raspberry Pi folder detections to the DropBox folder diningroom. Right after that all videos in the detections folder are deleted to make room for new videos and to avoid double-uploading onto DropBox. However care has to be taken to avoid that a video is being deleted while it is being saved – chances are not very high, but it can happen. In that case it would make the routine crash. The part after the semicolon in line 2 take care of that:

  1. List all files in the directory.
  2. Take all besides the latest in the list.
  3. Remove these (so the latest is not touched…).

One minute after midnight then (line 4) the diningroom folder on Dropbox is deleted with all saved videos in it. This is to free up space for new videos. So if you don’t save the videos somewhere else they will be gone at midnight everyday.
Line 5 then creates 5 minutes after midnight a new diningroom folder on DropBox ready for new videos with motion.

Set the Internal Time

You might also have to fix the internal Raspberry Pi time and time zone. I am using a Raspberry Pi 3 and fixed it with the help of this post.
Basically do the following:

  1. Stop the internal time:
  2. Set the right time-zone:

    …and choose ‘4 Internationalization Options’ to change the time-zone.
  3. Restart the internal time:

Running in the Background

One more thing I would like to mention:
If you want to run the program for long hours (maybe day and night) you need to make sure that it does not get terminated when the Raspberry Pi goes in standby mode.
For this you can make the program run in the ‘background’ with following call:

This will start the program in the background and all console output will go into a file named ‘nohup.out’. The program should now no more be terminated when you close windows or leave your computer untouched for a longer time.

So this post got pretty long – but I hope still useful if you want to use C++ for some video surveillance…

…and as usual: I am looking forward to your comments and thoughts.
Please leave your feedback in below form…