Archives for Raspberry Pi

Build a Raspberry Pi Vehicle Interior Monitor – Mobile Broadband

Posted by Graham Smith on July 20, 2017No Comments (click here to comment)

In this blog series I'm documenting my maker journey as I build a Raspberry Pi-based vehicle interior monitor (PiVIM). Please refer to the previous posts in this series for the story so far:

In this post I'm configuring PiVIM with mobile broadband connectivity. At this stage I don't yet know whether I will connect to PiVIM to query its status or whether I'll have PiVIM push notifications out (for example by SMS), however either way I do know I need some sort of connectivity. Setting the Raspberry Pi up as a WiFi hotspot would be a neat solution however since I need a range of up to 1 km I ruled this option out in favour of mobile broadband.

Let's Get Physical

My first task was to choose the physical mobile broadband device. A Google search for raspberry pi mobile broadband turns up quite a few hits for Huawei mobile USB dongles and what seem to be quite a lot of configuration steps to get them to work. However a friend recommended the ZTE range of USB dongles and I ended up buying a ZTE MF730M for testing purposes. This is a 3G unit and is well under under half the cost of the ZTE MF823 4G unit, however at some point I'll upgrade to the 4G version since it's more flexible. I was prepared for a painful experience to get it working but on an updated version of the latest Raspbian Jessie the ZTE MF730M just worked in true plug and play fashion.

In order to get the ZTE MF730M working I needed a SIM. I wanted to avoid a plan where monthly credit would be lost if it weren't used, since PiVIM won't get much use in winter but will get a lot of use in summer. The Three network have a PAYG SIM which fits the bill perfectly since the credit lasts for as long as it isn't used. In the UK these can be bought from Tesco for £0.99. You'll need to install it in to the dongle and leave it to activate (somewhere it can get a signal) before registering the mobile number on the Three website and adding credit.

Mobile Broadband Status

If all I wanted to do in this project was to use my mobile broadband dongle then the good news in the plug-and-play department would make for a very short blog post. I don't just want to use the dongle though, I want to display information about its status (network type, signal strength etc) on my Display-O-Tron control panel. The ZTE MF devices incorporate a web page (accessible at http://m.home) that displays status information, as well as functionality that allows the management of a phonebook and the ability to send SMSs:

It turns out that this web page gets its data via a REST API and it's possible to tap in to this API to retrieve information programatically. It's easy to see the API being used from a browser's developer tools (on the Network tab in the Chromium version that ships with Raspbian), however the good people on this GitHub site have taken the trouble to document some of the commands and have some example code.

I used their code as starting point and created a Python class to return the status of the mobile dongle via instance attributes. You can find the code on my PiVIM GitHub site as and there is an accompanying file that has code to put the class though its paces. The Python class minus a few docstrings is as follows:

I'm only returning three instance variables but clearly the code can be easily amended to return as many as are needed. One slightly ugly feature of my code is a hard coded response from the REST API to cater for when the mobile dongle isn't plugged in. My code should probably throw an exception if the mobile dongle isn't plugged in however when it is it's potentially using credit which I don't really want it to do for debugging purposes. So for the time being I'll live with my hack.

Screen Scraping for Remaining Credit

One piece of data that doesn't seem to be available from the REST API is the credit remaining on the SIM. In my case though it is available by logging in to the website with the SIM phone number and password and navigating to the Account balance page. There's no API in use on this website as far as I can tell so retrieving the actual value is down to screen scraping. Python has several libraries that can help here and I've been using requests and the BeautifulSoup class that's part of bs4. Long story short with this is that I've burned numerous hours trying to make this work and so far have drawn a blank. The problem is in authenticating properly with the Three website so that navigating to another page is successful. Although this aspect is work in progress I'm mentioning it because in a roundabout way I learned what I think are two great Python tips:

  • If you find that a Python library fails to install on Windows with the standard pip command it might well be that a compilation step failed. In this case you can try downloading an already compiled version of the library from the Unofficial Windows Binaries for Python Extension Packages site. (Note that AFAIK the 32/64-bit versions relate to the version of Python you are running and not whether you are running 32 or 64-bit Windows. Unless you have gone out of your way to install 64-bit Python you're probably running the 32-bit version.) Open a command prompt where you downloaded the file and type pip install followed by the first few characters of the library. Then use tab completion to complete the library name. In using this technique pip knows to install a library from your download rather than from the Internet.
  • Jupyter Notebooks are great for working with code on a ‘trial and error' basis where you want to repeatedly evaluate the output of a statement without having to run the whole program every time. For me this was working out which BeautifulSoup syntax would return the value of an HTML element that I was interested in:

    In the example notebook above, once the first four code blocks have been run I can repeatedly run the fifth block until I get the correct syntax for the statement that returns the authenticity_token. It's a real time saver over working in a more traditional code editor where the whole program needs to be run each time. You can find a good guide to getting started with Jupyter Notebooks here.

Hopefully I'll have time to pick up this screen scraping challenge again in the future. Meantime, if you are in the UK and fancy a crack at this then all you need to do is buy a £0.99 123 SIM from Tesco, pop it in your phone to activate over the Three network and then register the SIM on the Three website.

Tune in next time when I turn my attention to the hot topic of temperature measurement!

Cheers -- Graham

Build a Raspberry Pi Vehicle Interior Monitor – Screen Test

Posted by Graham Smith on July 2, 2017No Comments (click here to comment)

In this blog series I'm documenting my maker journey as I build a Raspberry Pi-based vehicle interior monitor (PiVIM). Please refer to the previous post in this series for an overview of the project:

In this post I'm getting started by configuring the Raspberry Pi with a mini display which will act as a control panel. The unit I chose to put through its paces was Pimoroni's Display-O-Tron HAT (DotHAT):

Why did I choose DotHAT? Actually for no other reason than I've seen it in action and I was impressed, and it's a reasonably low cost component given the functionality on offer.

Tour Starts Here

In order to make full use of the DotHAT you need to be aware that the HAT is actually a composite of several bits of hardware. The obvious component is a 16×3 character LCD display, and this is complemented by a six-zone RGB backlight, an array of six LEDs and six capacitive touch buttons (think joystick controls). Each component can be programmed separately as required—or not as the case may be.

Physical installation—as with all HATs—is straightforward as it just sits on the GPIO pins. An initial concern was whether the DotHAT would require the BCM 4 GPIO pin that I was planning to use for the DS18B20 temperature sensor however the DotHAT pinout shows that it's not used.

In order to easily control the components of the DotHAT Pimoroni have created high-level Python libraries that wrap the lower-level libraries that interface with the hardware—a function reference is provided here. Installation of these (and supporting) libraries is straightforward with just one line of code:

As always it's best to make sure your OS is up-to-date first. The above command will ask if you want to install the example code and I definitely recommend this so you can see for yourself the highly creative ways you can use the DotHAT. One of the examples is a fully-functioning Internet radio with a menu system driven by the capacitive buttons—very impressive. Do take the time to run the examples and explore the code as there is oodles of functionality to play with.

PiVIM Control Panel

In my PiVIM project I'm planning to use the DotHAT as a control panel to display information about PiVIM's status such as current vehicle temperature, mobile broadband signal strength, error messages and so on. At this point I don't know exactly what items I want to display, however I do know it will need to be fairly simple so I probably won't use the menu feature for example. Instead I will most likely write information as either Left or Right-aligned and to each of the three rows of the LCD (Top, Middle, Bottom), giving six locations to write to:

In order to simplify writing to the six locations I wrote my own module with functions such as message_left_top() and message_right_middle(). You can find the code in my PiVIM-py GitHub repository as PiVIM-py/pivim/ There is also a PiVIM/ module which contains some code to put control_panel through its paces. The core ‘message' functions are straightforward, however there is an issue to be aware of regarding writing a new message that is shorter than the previous message because you'll find fragments of the previous message will still be displayed. I envisage updating all six positions together in a loop and will get round this problem by calling the clear_screen() function before each iteration. If you are doing something different you'll need to code accordingly.

One interesting touch (pun intended) I added was to configure the left and right capacitive buttons to turn the backlight on and off respectively. With battery life in mind I then took this a step further by implementing a function that creates a thread which calls a timer to turn the backlight off after a delay:

The display_config() function (not shown above) also calls backlight_auto_off() to help conserve battery life.

Future Functionality

In the interests of YAGNI that's all the functionality I'm going to write for the time being, however I do have some ideas for the future. One exciting possibility is concerned with how I represent mobile broadband signal strength. The DotHAT supports the full ASCII character set of course, but intriguingly it also supports up to eight custom glyphs. So on the one hand I could use, for example, asterisks to represent signal strength. On the other though, I could have a go at creating the sort of glyphs that are used to represent signal strength ‘bars' on mobile phones. If you are already itching to have a go at this the documentation is here. Until next time—happy coding!

Cheers -- Graham

Build a Raspberry Pi Vehicle Interior Monitor – Overview

Posted by Graham Smith on June 20, 2017No Comments (click here to comment)

Over the past year or so I've been teaching myself whole new areas of learning based around the Raspberry Pi, including Linux, GPIO programming, basic electronics and Windows 10 IoT Core. I'm now at a point where I'm ready to build something that might be half useful, and I thought it might be helpful to someone if I blogged about my fledgling maker journey.

For my first project I'm going to build a Raspberry Pi Vehicle Interior Monitor—PiVIM. The idea is that on the odd occasion when we need to leave our dogs in the car for a few minutes, PiVIM will provide extra reassurance that all the ventilation and safety measures we've provided (windows partially open, tailgate open but secured with a Ventlock Tailgate Lock type device, reflective windscreen shade and so on) are actually working, through some sort of messaging to our mobile phones.

Important notice: The aim of PiVIM is only to provide extra reassurance on top of an already very cautious approach to reluctantly leaving dogs in the car for very short periods. Dogs die in hot cars!

With the sombre stuff out of the way, sure you can buy something but where's the fun in that? Making something from scratch offers an opportunity to learn a whole new set of skills, and in a new series of blog posts I'm planning to share my journey building PiVIM. In this first post I'm setting out the big picture—the features I hope to incorporate in to PiVIM and the developer tools I'll be using.

PiVIM Features

Here's a list of potential features that I'm considering for this project:

  • Temperature measurement. The key requirement for this project is to monitor the temperature of a vehicle's interior. A popular component for temperature measurement is the DS18B20. This comes as a small three-pin unit that looks like a transistor and also a waterproof version with the sensor embedded in a metal tube at the end of an attached wire. The waterproof version looks most useful for my project due to ruggedness and flexibility of being on the end of a wire.
  • Mobile connectivity. Since PiVIM will need to work in remote locations it will need a mobile internet connection. There's a cost to this of course, and I want to keep costs as low as possible. One of the problems with most mobile broadband plans is that they are based on a monthly data allowance and at the end of each monthly period any unused allowance is lost. Given that PiVIM might be used a lot in summer and very little in winter such a plan would likely be wasteful and uneconomic. Happily the Three network have a PAYG SIM where the data allowance lasts for as long as it isn't used. I'm planning to partner this SIM with either the ZTE MF730 3G USB dongle or the ZTE MF823 4G USB dongle, and both, if Google searches are anything to go by, should work with the Raspberry Pi.
  • Data access. Related to mobile connectivity is how to access the data that PiVIM generates. In addition to sending SMS alerts, the options that I'm considering are to store all data locally and make it accessible via a website running on the Pi, or to upload it to somewhere like Microsoft Azure and access it from there. Lots to research needed here, not least because although I have plenty of experience with Microsoft Azure right now I have no idea if it's possible to host a website on a Raspberry Pi that's accessible via a mobile broadband connection.
  • Battery powered. Although PiVIM could use a vehicle's 12V power supply via a USB adaptor the cabling would be messy and a dedicated battery feels more suitable. Tests with a RAVPower 22000mAh portable charger and a Raspberry Pi Model B with camera attached showed that the RAVPower could keep the Pi going for at least 36 hours (I stopped the test before the RAVPower was fully drained) so a unit like this feels like it will be a good choice. It would also be useful to have some power management system to monitor the battery's charge status.
  • Onboard display. I want to be able to see some basic information about PiVIM whilst it's running—mobile broadband signal strength, current temperature and so on. I've seen the Pimoroni Display-O-Tron HAT used for this purpose and was impressed, so that will probably be my starting point.
  • Power button. Raspberry Pis don't come with a power button and if left connected they will also gradually drain a battery even when powered down so I'll want some sort of solution to these problems.
  • Camera pictures. More of a nice to have rather than a necessity, but since the Raspberry Pi has a very handy camera module available as an accessory I might try and see if it's viable to access pictures over mobile broadband.
  • Robust case. The PiVIM internals will need to be well protected so some sort of robust case will be essential. It will need to be able to house the battery as well as the Pi, ZTE USB dongle and the Display-O-Tron. Current thinking is an electrical junction box such as the one here might be a good starting point, with the Display-O-Tron screwed to the exterior surface of the lid and connected to the Pi with something like the Pimoroni Mini Black HAT Hack3r.
  • Raspberry Pi model. I'll be prototyping on a Pi 3 Model B but might switch to a lower-powered board when it comes to building something that will be used out in the field.

Development Environment and Tools

I'll be starting off coding in Python, however a developer friend has very good things to say about developing with Kotlin for the Raspberry Pi so I'll probably try my hand at a Kotlin port once I have a Python version working.

In an ideal world I'd do all development directly on the Pi since there will be quite a lot of Python libraries that are talking directly to Pi hardware or to hardware attached to the Pi. In practice though I find that the development experience on the Pi doesn't give me what I want either in terms of performance or in the coding tools I want to use. Since I do a lot of work with Microsoft technologies my current development workstation is running Windows 10 and I use scp to push code out to the Pi which is running in headless mode on my local network. My configuration is as follows:

  • Windows 10 Pro with the Windows Subsystem for Linux (WSL) installed and a registry setting to ‘Open Bash window here‘.
  • I used to go to the trouble of giving my Pis fixed IP addresses so I could always be certain which one I was connecting to. I don't bother now and instead have Bonjour Print Services for Windows installed so that I can remote to a Pi using the hostname.local format. This works a treat in applications such as FileZilla and PuTTY. Unfortunately there is currently a bug in WSL which stops this from working. WSL is still in beta so hopefully this will be fixed soon.
  • I do find it's worth configuring SSH to use certificate authentication to avoid having to deal with passwords, and have the same certificate set up for both Windows 10 and WSL.
  • Python obviously needs to be installed—I just go for the latest version from the website here which also installs pip.
  • One of the issues with Python development is that if you don't do anything about it packages are installed globally. This creates problems if you need to create or edit Python code that needs a specific version of a package, or indeed Python itself. The solution to this is to use virtual environments courtesy of Virtualenv and (on Windows)  virtualenvwrapper-win. There's a great guide to configuring and using virtual environments on Windows here.
  • I'm using Git for version control and the Python version of PiVIM is on my GitHub site here.
  • My lightweight code editor of choice is Visual Studio Code. It's free and Python is fully supported with the help of Don Jayamanne's Python extension. The best way to start Visual Studio Code if you are using virtual environments is from the command line of a virtual environment using code . (make sure you don't miss off the period). Whilst you are at the command line make sure you install pylint (pip install pylint) in to your virtual environment and any other packages your code needs.
  • My heavyweight IDE of choice is Visual Studio. A free version is available and it's got a huge amount of support for Python via the Python tools. Whilst I don't use it on a daily basis for Python development it's great for remote debugging using the ptvsd package. Anyone who's used Visual Studio to develop .NET applications will love and appreciate the debugging experience and there are details on how to set up this awesomeness here.
  • I have FileZilla and PuTTY installed and have them configured to connect to my Raspberry Pi devices using SSH and certificate authentication. I have a bash script under version control on my Windows 10 workstation file system which I run from WSL (one of the handy things about WSL is that it can see the Windows 10 file system). The bash script uses scp to copy Python files to the Pi, after which I switch to PuTTY to run the code. A bit clunky but it works. (UPDATE: I've stopped using the bash script as it was too cumbersome. I now clone my code from GitHub to the Pi and and then in a PuTTY connection to the Pi—after having pushed code to GitHub—I run a command such as git pull && python3

That's it for now! Watch out for my next post in this series where I'll be getting stuck in to the details.