Monday, April 6, 2015

Fun with Raspberry Pi 2 and the buildfarm

Here's a picture of my two Raspberry Pi 2 boxes, both running headless and wireless.


One is running Raspbian, installed via NOOBS, and the other Fidora, a remix of Fedora 21 for Raspberry Pi 2. It turned out that Pidora doesn't work on the Raspberry Pi 2, a fact that is extremely well hidden on the Raspberry Pi web site.

I have set up test buildfarm animals on both of these. But something odd is happening. They are both getting intermittent failures of the stats regression test. Sometimes it happens during "make check", sometimes during "make installcheck" and sometimes during testing of pg_upgrade (which in turn runs "make installcheck").

These machines are not anything like speed demons. Far from it. But we also run other slow machines without getting this happening all the time. So I'm a bit perplexed about what might be going on.

Incidentally, if you want to play with one of these, I do recommend getting a starter kit from Amazon or elsewhere. It's probably cheaper than buying everything separately, and gets you everything you need to get started. Well worth the $69.99.

14 comments:

  1. Damn it! The possibility that there might be a bug in my code (due to the pgstat reworks in 9.2), and that someone else might find it, is unbearable. I just ordered a Raspberry Pi 2 to check what's happening ...

    ReplyDelete
  2. A faint memory says that when I did the same test on my Pis a few months (year) back, it was about the Make scripts assuming basic timeouts that the Pi's weren't able to keep up with (basically it was seriously low powered). And yes, the failures were intermittent and spread out at different places at different times (but most were using a component where the timeout was insufficient). All the best.

    ReplyDelete
  3. hamster is (one of) the slowest machine(s), and even these days builds are pretty stable:
    http://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=hamster&br=HEAD
    What is the failure you are seeing?

    ReplyDelete
    Replies
    1. If memory serves me well, the Make Check had parallel routines / tests, which didn't work too well with the Pi (so it intermittently failed), but I'd let Andrew confirm his recent findings as well.

      Delete
    2. Seeing this sort of thing:
      [55231754.498e:1] LOG: using stale statistics instead of current ones because stats collector is not responding
      [5523163b.4987:72] LOG: wait_for_stats delayed 153.721611 seconds

      Delete
    3. Just tested with ArchLinux ARM 3/4 times and things are working fine.

      Delete
    4. michaelpq, is that on an RPi, or some other ARM platform? Which ARM processor is it? Mine have armv7l.

      Delete
    5. That's a raspberry PI 2 as well, with ARMv7 and ArchLinux ARM.

      Delete
  4. Assuming you are talking about stats.sql I see it uses clock_timestamp() in combination with sleep(). I also see that none of the other SQL in the regress directory do. Wonder if what you are seeing is due to the fact that RPI does not have a real system clock?

    ReplyDelete
    Replies
    1. Possible, you may need to enable ntpd. That's what I do.

      Delete
    2. ntp is enabled, still seeing issues on the Fidora box. At this stage I'm inclined to suspect that the microSDHC card I'm using in it is just horribly slow. I have a SanDisk 32gb card available that I'm going to try instead. The Raspbian box is behaving now, I've really only seen one failure on it, and that was before I enabled ntp.

      Delete
    3. And just after I posted that comment HEAD failed on the Raspbian box with this:

      FATAL: could not create lock file "/tmp/pg_upgrade_check-oT6vZk/.s.PGSQL.57732.lock": No such file or directory

      Something weird seems to be happening.

      Delete
  5. FWIW, I did >100 runs of 'make check' on the RPi2 I have since last week and I see no such issues. In all cases I get "All 152 tests passed".

    I'd guess your SDHC card is pretty bad - can you do a simple "dd" test with it, for example? I'm using quite good "class 10" card from SanDisk.

    ReplyDelete
    Replies
    1. Yeah, I did exactly that this morning - replaced it with a 32GB class 10 SanDisk card, and magically the problem has disappeared. Thanks for testing.

      Delete