Executing Gummiworms The trials and tribulations of a grumpy curmudgeonly old git

28Jan/1224

Setting up a VM for Raspberry Pi development using Virtualbox, Scratchbox2 & qemu (Part 3)

I don't claim that the following is the best way to install scratchbox2, qemu and a seed rootfs and configure them to produce binaries that will run on the real Raspberry Pi hardware. I don't even claim it is the correct way but it works for me and I can almost do it in my sleep by now. The way I do things will probably seem a bit inefficient and in some cases just plain wrong but it's how I work and hopefully they are easy to follow and adapt to your style of working. I also make no guarantees that it will actually work for you at all. Hopefully it will but if it doesn't sorry but oh well. I made the decision when I first started building the development vm that I would place all the required software under the users home directory rather than installing it globally as

  1. Installing in subdirectories in the user's home directory makes it easy  to keep things organised.
  2. It makes things almost idiotproof when you want to upgrade the ARM toolchain, scratchbox2, qemu or change the seed rootfs etc. as it's pretty much just rename the old directory, create a new directory and if neccessary rerun sb2-init.
  1. Open a terminal
  2. If you haven't already got git & wget installed. Install them now. On Ubuntu or Debian the command to install them is
  3. Create a directory for temporary use. I personally call it hold and change to it.
  4. Download scratchbox2 and qemu from their respective git repositories
  5. Download the codesourcery ARM toolchain. I have been using the 2011.03 version successfully and I believe it was the last released version before codesourcery was bought by Mentor Graphics who seem to have closed sourced the more recent releases of the toolchain.
  6. Then download an ARM rootfs. Eventually we will be able to use the official Raspberry Pi standard rootfs but until that time any ARM based one will do. As I believe that the official Raspberry Pi distribution is going to be Fedora we'll download a Fedora one. In this case we use one I found on the fedora-arm mailing list. http://lists.fedoraproject.org/pipermail/arm/2011-December/002386.html
  7. If you now do a directory listing you will have a terminal window that looks this
  8. Now create some more directories.
  9. Untar the rootfs The reason we extract the seed rootfs using sudo is that there are some special files that a normal user cannot create when untarring a tarred rootfs. Lots of filenames will scroll before your eyes as the seed rootfs is untarred into the rootfs subdirectory. This will take quite a while so it might be a good time to go and get a drink or take a bathroom break.
  10. Extract the codesourcery ARM toolchain into the raspberry_pi_development directory.
  11. Do a directory listing of the raspberry_pi_development directory and it will look like the image below.
  12. If you haven't already got the SDL and ncurses libaries and  headers installed. Install them now. In Ubuntu and Debian
  13. You should also install autoconf, fakeroot and realpath. Fedora users will need to find their own copy of the realpath source as it is not included in the distribution. I found a copy that works on stackoverflow. In Ubuntu and Debian change to scratchbox2 directory and run the autogen.sh script.
  14. then run makeYou can ignore all the warnings that will scroll passed and when it finishes the screen will look something like the one below. The interesting lines are the final 4 or 5 which should basically say the same thing that the screenshot does.
  15. Scratchbox2 is now installed and we now need to build qemu so cd to the qemu directory
  16. To use scratchbox2 you only need to build qemu usermode for ARM, however, I find it useful to also build the ARM system emulation as well (I use the qemu full system emulation for some little hacks and tricks that are beyond the scope of this howto but I will write them up at a later date along with how to use scratchbox2 with real hardware once I have the process down pat & actually have a real Raspberry Pi to test on).
  17. Now that scratchbox2, the toolchain, qemu and the seed rootfs are installed we just have a few more steps before we can actually use the VM for compiling software. First of all we need add the scratchbox2 and qemu bin directories to our PATH environment variable. You can do this as a single export statement but so it's clear i'll do it as two. You will also want to add the previous two lines to your .bashrc file. Start your favourite editor and open the .bashrc file and add them to the bottom and in the case of nano save the file using ^x y <enter>. If you are using vi the it would be <esc> :wq
  18. Now that you have added scratchbox2 and qemu to your PATH (and in .bashrc as well) check that it works.The version numbers returned might be different but that doesn't matter it's the fact that you actually got the version numbers that confirms that the PATH environment variable is set correctly.
  19. change to the rootfs directory
  20. before we can use the seed rootfs inside scratchbox2 we need to change the owner, group and permissions so that it is read/writable by our non-root account. When I installed Ubuntu into the VM I was asked to make a default user. In my case I called it raspberry and gave it a password of password. You might have called your default user something different replace raspberry in the following lines with your user name.
  21. There is just one more thing to do before we can start using scratchbox2. We need to initialize it. While inside the the seed rootfs directory. In this case rootfs_f14sb2-init actually has a lot of options you can use but in most cases they just complicate matters and for our needs the above command line is good enough. What it actually means is configure scratchbox2 to create a target called raspberry and use the toolchain binaries that we have installed in
    $HOME/raspberry_pi_development/arm-2011.03/bin

    As you get more familar with scratchbox2 you might want to experiment with things such as having multiple targets with a single scratchbox2 installation etc.

  22. After a short wait your screen will look something like this
  23. You are now ready to start using scratchbox2 to compile software. However, I do a few more things just to make life easier that you might also want to do. I create a directory called $HOME/build, install an ssh server and apache2 and put a symlink of the build directory in the apache2 directory tree. This allows me to keep all my ARM binaries an source seperate from anything else and lets me get them out of the VM easily although I could also use shared folders but I prefer using a webserver as then any machine on my network can access them. I got a bit bored doing the screenshots and cutting out the relevant parts so here is a video of me doing this final bit of setupYouTube Preview Image

If you have got this far you are now ready to start building software for the Raspberry Pi. As a test I create a C hello world program and check that it compiles and runs both in the host os and also inside scratchbox2 and if that works then I check that the seed rootfs's package manager works. If both do then i'll start using the vm. Again because I am bored with doing screenshots here is a short video of that process.YouTube Preview Image

When and if I get time I might do some more blog posts about some advanced scratchbox2 and qemu usage, tips and tricks but for this post i say

THE END

Share
Comments (24) Trackbacks (2)
  1. Fantastic effort! Thank you for the detailed and visual instructions.

    • thanks. hope you find them useful. the information is out there already but i think this is the first time it’s been done with pretty graphics and a couple of videos :)

  2. That’s exactly what I was searching for, thanks!

  3. Certain irony about Raspberry pi bringing computing to a new generation with this demonstrating nicely unfathomable (and as retyped) typo-ridden command lines and all that’s bad about computing.

    Mind you…I guess I was typing in programs blind from magazines on a ZX81 I guess it’s no different!

  4. Has somebody managed to use sb2 and qemu from apt-get? I am getting a

    checking whether the C compiler works… configure: error: cannot run C compiled programs.

    error during sb2-init

  5. yes using apt-get can/does work. the first time many moons ago I used scratchbox1 & 2 I installed them via apt and they worked but I do remember having some problems a few times. If you look at this page http://russelldavis.org/page/2/?s=scratchbox it might help with the sb2-init command. if not email me or use the contact form and we’ll see about getting you sorted.

    • Hm, I’ve tried `sb2-qemu-arm`, `qemu-arm` and `arm`, none of wich worked…

      The weird thing is it lets me compile the stuff and `file hello.arm` returns

      hello.arm: ELF 32-bit LSB executable, ARM, version 1 (SYSV), dynamically linked
      (uses shared libs), for GNU/Linux 2.6.16, not stripped

      but running the file `sb2 ./hello.arm` segfaults:

      qemu: uncaught target signal 11 (Segmentation fault) – core dumped
      Exit reason and status: signal 11

      • that sounds like an old qemu version. i used to get that a lot. what does qemu -version (or qemu-system-arm -version or … ) give you?

  6. Hey! Nice work, but why don’t you share the virtual machine ready for making software. I think a lot of people will appreciate it. Thanks!

    • the vm is about 4 posts previous. i put the instructions up because i’m not going to have time to keep up with any changes to sb2 or qemu or rootfs… for the next few months and it takes ages on my upstream to upload.

  7. wow, nice stuff, but please use or around commands so i can cut and paste them

    • there were actually a several reasons why the command lines were not done as copy&pasteable text. 1. i’m mean, 2. I wanted to sort of hark back to the type-in days of old 3. i’m really mean 4. i think people remember better if they have to type it in themselves.

      • Even if not for cut and paste simple text would be much easier to read than those screenshots (and much less work vor you to just copy the text than to make and edit all those screenshots). As I have only one monitor I had to move the windows around all the time to be able to see the instructions and to be able to see what I am typing. If I have instructions as text I just print them out and can read them while typing.

        Anyhow: Thank you for the detailed instructions! Now that I have set up the VM I’l have to find out what I can do with it (apart from playing Mahjong ;-))

  8. i have a problem when compile scratchbox2:

    [CC] utils/sb2-show.o
    [LD] utils/sb2-show
    utils/sb2-show.o: In function `call_sb2__load_and_execute_lua_file__’:
    /home/gena/hold/scratchbox2/utils/sb2-show.c:93: undefined reference to `dlsym’
    utils/sb2-show.o: In function `call_sb2show__map_path2__’:
    /home/gena/hold/scratchbox2/utils/sb2-show.c:75: undefined reference to `dlsym’
    utils/sb2-show.o: In function `call_sb2show__execve_mods__’:
    /home/gena/hold/scratchbox2/utils/sb2-show.c:82: undefined reference to `dlsym’
    utils/sb2-show.o: In function `call_sblog_vprintf_line_to_logfile’:
    /home/gena/hold/scratchbox2/utils/sb2-show.c:107: undefined reference to `dlsym’
    utils/sb2-show.o: In function `main’:
    /home/gena/hold/scratchbox2/utils/sb2-show.c:793: undefined reference to `dlopen’
    /home/gena/hold/scratchbox2/utils/sb2-show.c:800: undefined reference to `dlsym’
    utils/sb2-show.o: In function `call_sb2__lua_c_interface_version__’:
    /home/gena/hold/scratchbox2/utils/sb2-show.c:97: undefined reference to `dlsym’
    utils/sb2-show.o: In function `call_sb2__set_active_exec_policy_name__’:
    /home/gena/hold/scratchbox2/utils/sb2-show.c:89: undefined reference to `dlsym’
    utils/sb2-show.o: In function `call_sb2show__get_real_cwd__’:
    /home/gena/hold/scratchbox2/utils/sb2-show.c:101: undefined reference to `dlsym’
    utils/sb2-show.o: In function `call_sb2show__binary_type__’:
    /home/gena/hold/scratchbox2/utils/sb2-show.c:70: undefined reference to `dlsym’
    utils/sb2-show.o:/home/gena/hold/scratchbox2/utils/sb2-show.c:113: more undefined references to `dlsym’ follow
    collect2: ld returned 1 exit status
    make[1]: *** [utils/sb2-show] Error 1
    make[1]: Leaving directory `/home/gena/hold/scratchbox2′
    make: *** [regular] Error 2

    I try to add “-ldl” to LDPATH but its cant fix this problem

    • that’s a known problem with ubuntu 11.xx. Basically ubuntu 11.xx are screwed up (whatever happened to doing some Q&A? Use 10.04lts or another distro instead

      • It’s not screwed up… It’s a change in GCC that requires that you do your GNU Make and other build files when linking correctly. –as-needed is turned on by default and if you’re putting your .so’s in front of the .o files in your link order, it skips things. If it’s anything, it’s more on FSF’s (and Ulrich Drepper’s) head if you’re going to claiming “do some QA”…

        Easy fix is to look at the make files where this is happening within Scratchbox and move the .so positions to dead last in the order and it’ll just simply work. Technically, you’re supposed to do that in the first place…

        Now…you want to gritch about lack of QA work…trying to sort out what Ubuntu busted with 64-bit and Scratchbox so I can get back to trying to get my projects going in anticipation of laying my hands on a handful of the second batch of Model B’s. It’s very likely they busted something; but it’s a wierd thing.

  9. Out of interest, if I made stripped the GUI from Ubuntu/Debian after setting this up, would you expect me to see any long term issues with the ARM environment? (I don’t see any reason why, but thought I’d ask to be safe)

    • i can’t see why not. the toolchain, scratchbox2 & qemu don’t need X
      qemu can be made to link with SDL but that can use the framebuffer so shouldn’t be a problem

  10. nice job ! works perfect !
    save me a lot of time – many thanks

  11. Great job! Works well on 12.04 LTS 64-bit. The only difference with x64 is that you need to include the 32-bit libc6-dev-i386 using:

    sudo apt-get install libc6-dev-i386

    Otherwise you get /usr/include/features.h:323:26: fatal error: bits/predefs.h: No such file or directory while compiling. See http://ubuntuforums.org/showthread.php?t=1877944 for a more thorough thread on this.

  12. nice job!

    If someone is trying to implement it with a firewall in the middle you can use http:// in place of git:// . And maybe you have to change qemu to http://git.qemu.org/git/qemu.git (note /git/)


Leave a comment