Get Me To Town!

Ever since I saw this project (http://digitalfantastico.blogspot.co.uk/2013/01/a-big-bite-of-raspberry-pi-having.html) by Gareth James at Brighton Hackspace, I’ve wanted to make my own display like it. It’s a display that shows when the next trains are from his local station to get into Brighton.

I need to get from Beeston to Nottingham, where I have more choices than just one train. There are two different bus stops (both with different routes) and a train station I could use. Working out which one I should use at a particular time requires looking at three different webpages, which is a bit of a faff.

The basic principle of the display is to scrape website(s) for the relevant information and display it in a easy-to-glance-at format. Gareth’s display used a RaspberryPI to perform the scraping and a TFT screen for the display.

Since I have an HP Touchpad that I barely used, I decide it was finally time to put Android on it (Cyanogenmod), and get into the world of app development. Android devices have a “Daydream” mode (a bit like a screensaver) that can come on when charging. To my (limited) knowledge, they’re just applications that declare themselves as daydreams. This was what I chose to write, since I could put the tablet on its wireless charger and automatically display the application, and then just pick it up when I need to use it normally.

The website I chose to scrape for the bus information is http://www.nextbuses.mobi. For trains I used http://www.traintimes.org. The “Terms” page asks you kindly not to use it for apps, but I hope that my one-request-every-10-minutes-or-so-on-just-my-tablet application won’t cause an issue.

When the application level update manager decides it needs to perform an update, the data flow from website to the screen is:

GetMeToTown dataflow and components

GetMeToTown dataflow and components

  • An asynchronous URL thread is started that pulls the raw HTML.
  • The HTML data is passed to a class that uses TagSoup to scrape it and produce strings containing the pertinent information.
  • The strings are passed to a handler class that produces a list of individual journey objects.
  • The journey objects are passed to a manager that collates all the journeys from each website and sorts into time order.
  • The application requests journeys from the manager and decides which ones to display (based on user-configurable time and display settings).
  • The display is dynamically re-generated if the number of displayed rows needs changing.

Every 30 seconds, the screen is updated with how much time is remaining for each journey.
This 30-second tick also allows the journey manager to discard old journeys and to decide when to make a request for new website data.
A new request is made each time an item is removed from the list or when 15 minutes has elapsed since the last update. This means that most of the time a lot of data is pulled for journeys already in the list. The reason for doing it this way it means the data is “more live” – in case of delays/cancellations/breakdowns to services.

The application all runs off a state machine (I love state machines. They make things sane.) using stateless4j (https://code.google.com/p/stateless4j/), a Java port of the C# “stateless” project (https://code.google.com/p/stateless/).

The 30-second tick is kicked off by a handler registered with the Android API. The tick itself just passes an “update display” request into the state machine, or starts an update if the update manager decides that one is required.

Here’s what the screen looks like on a typical weekday morning:

The GetMeToTown display

The GetMeToTown display

The screen displays eight items, each with an icon showing the transport type (the British Rail symbol for trains, the relevant bus route logo for the four different buses I can take) and the time the service departs. The time is in “minutes remaining” format for journeys in the next 20 minutes, and the departure time for journeys further in the future.

It takes me about 5 minutes to walk to the stops, so the time turns red with 10 minutes to go. The journey is removed from the screen with 4 minutes remaining, since I certainly won’t make it after that.

Most of the numbers (number of journeys to display, update times, journey discard and highlight times) are user-configurable with a Settings activity.

Posted in Computing | Tagged , | Leave a comment

SimCity: Reminiscence Edition

If you read this for electronics stuff, prepare for disappointment. Because this is a sleep-deprivation induced rant about a video game.

SimCity: What the hell happened to you?

Every now and then I feel like going back and playing SimCity. When I was the right age to really appreciate the game, SimCity 3000 was the latest one at the time. It had pleasant, pleasing graphics and soundtrack, lots of scope and didn’t take itself too seriously. The cities you built could grow to enormous size, enough to push the 900MHz Athlon I had at the time to its limits when drawing the whole thing.

Crucially, it wasn’t SimCity 4. I got it on Steam a while back. Argh, that was a mistake.

When I play SimCity, I want to build and run a city. I don’t want to drive around it (that’s what GTA and Driver are for). I don’t want to care about individual Sims in it (more than any others).

What I really want is a more realistic “local government simulator”. For me the best bit about SimCity, apart from the long-term building, was choosing all the progressive, green, lefty policies and having them work.

What I want from SimCity

Features I would love to see in a better SimCity would be:

More realistic building timescales

When you zone land for buildings in the game, it takes time for buildings to appear. If the economic conditions aren’t right, they may not appear at all. I want the same thing for other building. You want three miles of road? 8 months, and the cost is an estimate. You want a new water pipe under that industrial zone? That’s lots of digging. One year minimum, and the economic output of that area will fall. Oh, you need that pipeline now? Too bad, should have planned earlier. You’ve changed your mind about the construction of that new motorway? It’s going to cost time and money to remove what we’ve started.

Realistic limits to what you can build

I was always a bit dubious about building my own power plants. Isn’t that a national government/utilities thing? Since when does a town with a population of 5000 just decide to build a coal fired power station?

Towns aren’t (typically, at least)  owners and operators of their own utilities. However, the local government gets to decide who gets to build what, and where. I want the Sim equivalent of EDF asking to build a new nuclear plant, and the likely impacts presented. I want to be able to refuse, and then I want the chance that the national government forces me to have it anyway and my citizens and I have to cope with that.

The feeling that I’m part of a bigger country

On the subject of national government, I want much more interaction with them. A huge part of local government is balancing the needs of your citizens with the wider national (and indeed global) interest. For instance, the government is going to put a high speed railway through your countryside. Do you support it or not? What do your citizens think? Will it stop in your city, and if it does, will it be a boon or a drain?

On the subject of the countryside, I want something akin to greenbelt laws. Why do my Sims not kick up a fuss when I fill in that lake or raise a forest to the ground? Maybe I should only be allowed to zone a new commercial block if I agree to build a load of new parks – with penalties if I don’t. Maybe the government will decide to stop me zoning altogether in some places. Who knows, but I do know that local authorities can’t just decide that a bit of land will be for houses now – as the regular e-mails from my MP will attest.

The feeling that I’m part of a wider economy

You already get a fair few economic choices in the game, but it’s all fairly local. I want to see the impact of the wider economy. You got to trade water, power, waste with the surrounding towns and cities, but there was never a sense of a fully developed economy. I want the town up the road to get a new factory, pulling activity away from my town and causing massive traffic problems. I want my local economy to be affected by the national one. Employment and investment as part of a national and global whole.

In short…

I want the plans I make to be forced to take time and the wider world into account, not just – as was most often the case in SimCity – money.

That sounds simple enough. Over to you Maxis.

Aside | Posted on by | Leave a comment

Cheap-ass Logic Probe in 15 minutes

I had a sudden need for a logic probe. Here’s how to build one in 15 minutes.

Element 14 pen

Element 14 pen

Get an Element 14 pen. IT MUST BE FROM ELEMENT 14. No other normal biro will work.

Cut off the clicker end with a junior hacksaw and discard the useless innards.

IMG_20140409_193523

Drill a 2mm hole in the side. Stick a black wire through and terminate with a crocodile clip.

IMG_20140409_193557

Cut a single header off a header strip. Solder on a red wire.

IMG_20140409_193619

Pick a soldering iron you don’t particularly care for. Melt the plastic of the header until it fits in the end of the pen.

IMG_20140409_194327

Solder an LED and appropriately valued resistor onto the red and black wires. Use heatshrink to cover up your soldering crimes.

IMG_20140409_195457

Use polymorph to secure the LED into the end of the pen.

Hey presto! Cheap-and-dirty logic probe.

Posted in Electronics | Tagged | Leave a comment

Hacking Ugandan Toilets http://nicnaknoe.wordpress.com/2014/03/09/hacking-for-ugandan-toilets/

Posted in Uncategorized | Leave a comment

PCB Software Choices

I have an imminent decision looming, and I’m hoping that spelling it out in a blog post will help frame my thoughts. <lie>It’s a tough one, with serious short- and long-term consequences either way.</lie>

Do I switch to KiCAD for PCB layout, or jump to a paid-for Eagle version?

I know, it’s up there with the biggest of ethical dilemmas, like that one about killing people on train tracks (I forget the details). But have no fear, let’s first state the reason why I need to think about it.

Currently, I use the free version of Eagle, the popular PCB package from Cadsoft (and heavily promoted by Element14). This is fine for me. I don’t need more layers or a larger board size. BUT, I’m hoping to start selling some stuff on Tindie. And the license for the free version of Eagle doesn’t let you use it for commercial purposes. I think that clause is aimed at businesses making hundreds of boards, not hobbyists who will be lucky to sell one. But still, I feel that Cadsoft are doing good by providing a free version of their software, and it would be morally wrong to exploit that good for profit.

KiCAD is an open-source PCB package that’s also very popular. It appears to have come on leaps-and-bounds recently, and (excitingly) CERN have put someone on it full-time. That’s the kind of awesome stuff CERN does. Moving to KiCAD would eliminate the licensing issue.

Here’s all the pros and cons of this momentous decision:

Stick with Eagle Move to KiCAD
Pros
  • No new learning curve
  • Sparkfun/Adafruit etc. libraries available
  • No need to port old PCBs to new software
  • No cost for selling stuff
  • Supporting open-source software
Cons
  • ~£70 cost for the right license, which I might not ever make back
  • Paid version still has limits on layers and board size
  • Closed-source software
  • MASSIVE learning curve
  • If I want to sell an old board, need to port it across to KiCAD
  • Seems to be fewer libraries/standard components than Eagle

Looking at this, all the cons of the “stick with Eagle” option after fairly minor compared to those for KiCAD. Closed-source isn’t really that big of an issue for me, I can afford the £70, and I don’t expect that the board limits will be an issue in the long term. My only concern is that the £70 will be wasted if I don’t sell much on Tindie.

After all this, it looks like I’m sticking with Eagle. £70 is less than ten hours at a UK living wage (reckoned at £7.65 per hour). I’d waste at least that much time learning KiCAD before I’d even start using it. While I don’t really think measuring my free time that way is completely fair or reliable, it’s a useful metric. And in this case, I think it’s a valid one. Either way I’m spending money or time, and I think my time is worth more here.

You have been hearing my brain think.

Posted in Electronics, Ranting | Tagged , , | Leave a comment

Mains Frequency Display

I seem to be making a lot of displays lately. Here’s another one.

Design Concept

I’ve had this planned for a while. The concept is to display the frequency of the UK mains electricity supply. This is meant to be 50Hz, but the frequency actually fluctuates around this value, as a result of changes to supply and demand.

If the frequency drops, it’s because demand has risen and/or supply has dropped, so generators have to work harder. This slows then down a bit, and the control systems take time to speed them back up.

If the frequency rises, it’s because demand has falled and/or extra supply has come on, so generators have to work less. This increases their speed a bit, and the control systems take time to slow them down.

So the frequency is constantly rising and falling. You can see a real-time graph of the last 60 minutes on the national grid website, and a nice real-time analog-style meter at the Dynamic Demand website.

I wanted to make a display that could show the mains frequency to 3 decimal places. I’d be using the same seven-segment display modules that I used in my UNIX clock, so all I had to do was design the part that would work out the frequency.

How to work out the frequency

There are lots of techniques to do this. My choice was to count a fixed number of mains frequency periods and time how long that takes to occur.

Here’s a drawing of the concept: The sine waves represent the mains. A high-frequency signal is run in parallel and used as a counter. The count after a fixed number of cycles is inversely proportional to frequency.

The concept of measuring varying low-frequency signal.

The concept of measuring a varying low-frequency signal.

For example, if the frequency was exactly 50.000Hz, and I counted 200 periods, that would take exactly 4 seconds. If the frequency was actually 50.001Hz, 200 periods would only take 3.99992 seconds.

That’s not a lot of difference! In order to time that accurately, I need a high-accuracy, high-stability timing source. The RC oscillator in the microcontroller wouldn’t do. I would be using an ATTINY84, which has an accuracy of only +/- 10%, or +/- 1% after user calibration. Nor could I use an external crystal, without paying a LOT of money for a high-stability one.

Luckily, there is one class of cheap(ish) ICs that have high-accuracy, high-stability oscillators in them, and that’s real-time clocks. The DS3231 RTC chip, which was also used in the UNIX clock, has a +/- 2ppm 32.768kHz output. By counting pulses from this, I can determine the time very precisely.

Maths!

After doing some maths, I figured that I could count for 100 mains cycles (2 seconds at 50Hz) and get my required accuracy.

The formula to translate count to mains frequency is

f=\frac{1}{\frac{\frac{C}{32768}}{N}}

where N is the number of mains cycles counted and C is the number of 32768 Hz cycles counted.

I re-arranged this, and added a 1000x multiplier in order to only use integer maths, so 50.000Hz would actually be stored as 50000. This is much nicer for implementing on a microcontroller.

f = \frac{32768 * N * 1000}{C}

Because we’re doing integer maths, we need to account for rounding, so the final formula is

f = \frac{(32768 * N * 1000)+\frac{C}{2}}{C}

For example, a count of 65724 would equate to a frequency of 49.857Hz, stored as 49857. A count of 65723 would equate to a frequency of 49.858Hz, stored as 49858.

So, a change of one count represents a change of one significant figure (of the scaled frequency), exactly what I want.

The frequency value is sent directly to the display, so the frequency shown is updated every two seconds.

There are two LEDs on the control PCB to display the up/down trend of the frequency. The frequency is stored in a ring buffer once a minute. A ten-minute rolling average is calculated, and the difference between the first and last readings is used to work out the trend. A difference of more than +/- 0.02hz is counted as a significant trend.

In order to find the “best” algorithm for this, data from the National Grid real-time display was parsed with the Beautiful Soup python module. I just experimented with various guesses until it looked about right.

Putting it all together

I tested the code on an Arduino with an opto-isolated mains input before designing the final circuit. The electronic design draws on a lot of previous work I’ve done. It has:

  • The mains-frequency input from my Days-Accident-Free counter.
    • (This clamps a 9V AC signal (from a small wall-wart transformer) to the 5V supply rails)
  • The display connection and DS3231 RTC from the UNIX clock.
  • A standard ATTINY84 microcontroller from a host of other projects.
The system block diagram

The system block diagram

I was able to reduce the board size down to be the same size as a seven-segment display, which was nice. I ordered the boards from Ragworm, and everything worked first time, which was nice.

The only modification I made was to add a small “heatsink” to the 5V regulator. The input to the system is 9V AC from a small adaptor. Because the regualtor runs all 5 displays, it’s dumping about 1W of power into a rather small package. It gets pretty hot, so I added a heatsink made of lots of solder on a bit of stripboard. A bit of a hack, but it does help keep the regulator cooler than it would be.

The control PCB, with LEDs, microcontroller, power supply and "heatsink".

The control PCB, with LEDs, microcontroller, power supply and “heatsink”.

Finally, I made a laser-cut case (as is practically standard for my projects). It was an adaption of the UNIX clock casing. For the up and down trend LED symbols, I tried using crayon wax melted into two laser-etched arrows. This worked pretty well, so I think I might use this technique on future projects.

The completed display, showing a "down" frequency trend.

The completed display, showing a “down” frequency trend.

The frequency trend of interest

The frequency trend of interest

Posted in Electronics | Tagged , , | 10 Comments

How the Arduino IDE tries to be too helpful

A Problem

The Arduino IDE tries too hard to be nice.

Here’s some Arduino code from a recent project of mine:

enum state_enum
{
    STOPPED,
    ACCEL,
    RUNNING,
    DECEL
};
typedef enum state_enum STATE_ENUM;
static STATE_ENUM currentState;
static void setState(STATE_ENUM newState)
{
    /* Extra code omitted for clarity */
    currentState = newState;
}
void setup()
{
    /* Extra code omitted for clarity */
    setState(STOPPED);
}
void loop()
{
    /* Extra code omitted for clarity */
    setState(RUNNING);
}

We’ve got a state variable that can be in one of four states, and a function to change state.
Looks OK, right? Well, try compiling it and you’ll get the following error:

sketch_dec04a:3: error: variable or field 'setState' declared void
sketch_dec04a:3: error: 'STATE_ENUM' was not declared in this scope

Which essentially means that on line 3 of my code, the compiler has found a datatype called “STATE_ENUM”, in the setState function, and doesn’t know what that means. But how can this be, when the function isn’t defined here, but about a dozen lines further down?

Turns out that the Arduino IDE (as part of its pre-compile process) scans your code looking for functions, generates prototypes for them and sticks them just above your actual code.

So, this line:

static void setState(STATE_ENUM newState);

Was being inserted above the declaration of the STATE_ENUM type, which causes the compiler to quite rightly say that it has no idea what STATE_ENUM is.

I can see why Arduino have done this: it means inexperienced programmers can put functions anywhere in their sketch and not have to deal with prototyping them.

Possible Solutions

So, how DO you use user defined types in your Arduino sketch?

Only use standard types

Well, the messy way is to avoid using functions that return or use those types. So, I could have altered my setState function to be

static void setState(int newState)
{
    /* Extra code omitted for clarity */
    currentState = (STATE_ENUM)newState;
}

This is a bit messy, as it hides the intent of the function input a bit.

Use a header file

Another option is to put the enumeration and typedef in a header file and #include it. The file is included before the auto-generated prototypes, so the compiler is happy.
This again is a bit messy, and not in the spirit of hiding implementation detail. If nothing else needs to know about your datatype, it shouldn’t be in a header file.

Hide the function from the auto-generator

Finally, the Arduino website build process page gives us a clue how we can solve this:

“Also, this generation isn’t perfect: it won’t create prototypes for functions that have default argument values, or which are declared within a namespace or class.”

So, we can make the function have a default argument:

static void setState(STATE_ENUM newState = STOPPED)
{
    /* Extra code omitted for clarity */
    currentState = newState;
}

…which will successfully compile, but now risks being called without an argument, which has the potential to do harmful things.

For the uber-paranoid, you could add an extra “invalid” value to the enumeration, make this the argument default, and test for this value right at the start of your function:

static void setState(STATE_ENUM newState = INVALID_STATE)
{
    if (newState == INVALID_STATE) { return; } // Do nothing if called without a state!
    /* Extra code omitted for clarity */
    currentState = newState;
}

The main disadvantage of this is that ALL the arguments after the defaulted one also need to have defaults. For example, this won’t compile:

static void setState(STATE_ENUM newState = INVALID_STATE, int anotherArg)

Because anotherArg needs a default value. This might be annoying.

The REAL solution

Ultimatley, the real solution is for the Arduino IDE to allow you to turn off automatic prototyping. Interestingly, an “Arduino for Visual Studio” application does exactly that.

There are LOADS of ways to do this, but I think a nice work-around would be a comment-style function decorator. Something like this:

static void setState(STATE_ENUM newState)/*NOPROTO*/

The Arduino build process would recognise the NOPROTO comment and suppress output of a prototype for that function.

This would avoid adding extra options to either the IDE or the preferences.txt file, and allow individual control of which functions are affected. A downside is that if you have a LOT of functions like this, the code gets littered with /*NOPROTO*/ everywhere.

What do I do?

Well, for the specific project above, I just modified the function to use a standard type.

I don’t tend to use the Arduino IDE much, so I think what I do in future will depend on the application.

Remember the mantra: “Software development IS decision making.” Get it tattoo’d on your eyeballs.

Posted in Computing | Tagged , | Leave a comment