This is my first post about the Raspberry Pi Pig-Tank -- a project my son and I have been working on for a little while. The project is still incomplete, but very functional. When it finally feels "1.0" we'll make a YouTube video that details the project further, but in the meantime I thought it'd be potentially useful to others to see our progress thus far.
Eventually I'll also get a post together that outlines measurements, part numbers and has detailed photos, but that's a ways off.
Motivation and Goals
See, I've been fiddling with Raspberry Pi for several years now, but have largely used them simply as small, lower-power computers. Anything involving more interesting hardware-wise would have me resorting to a microcontroller. I was interested in using a Pi as a control system just for the heck of it, but lacked the inspiration for an interesting project.
After building a simple RC tank kit from Radio Shack my son mentioned the idea of mounting a camera on it. The kit itself lacked any real intelligence. It was pretty much just a chassis with treads, motors, motor drivers, IR receiver (this kit is technically not RC because it's not *Radio* controlled), battery box... standard stuff. If we were going to get a camera involved we were going to need to get some proper computing power on there. Considering I have more Raspberry Pi and camera boards than I can shake a stick at it seemed perfect for the project.
As our conversations continued it was clear that we'd have the opportunity to extend the range of the vehicle. A Pi can easily interface with various wireless communication systems: XBee, WiFi, GSM... To keep things simple to start we agreed that WiFi made sense so we could at least control the vehicle around the house.
Also, it seemed reasonable that we should ditch specialized remote controls and go with something web-based. That would allow us to use our laptops, tablets and phones to control the vehicle. Not only are they easy for dealing with the video output of cameras but it also increases the cool factor a bit.
We agreed that our end goal should be to pilot the vehicle 0.6 miles from our house to the local grocery store and back just as an arbitrary measure of awesomeness (we'd obviously have to go beyond simple WiFi to get it done). If anything I pushed for that goal just to get it in his brain that remote control can actually be *very* remote. Maybe it would give him some additional appreciation for the wonderful work that's been done on massively-frickin-ridiculously-remote-controlled vehicles (MFRRCVs) like the great work NASA has done on the mars rovers. If we can take this thing from tens of feet to hundreds to miles the possibilities are endless!
So I set about getting materials together to build the bloody thing.
The plan was simple for the structure and drive. Use the chassis, treads and motors from the kit for the structure and drive system. From there holes could be drilled and components fastened.
The control system will be a Raspberry Pi with a USB WiFi dongle. Due to favorable power consumption characteristics a Model A was chosen (the Model A runs sub-200 idle while the B runs > 450).
We went with these 7.4V (both 1000 mAh and 2200 mAh versions fit in my case) LiPo batteries I had laying around. Although the Pi and motors were all fine with 5V these batteries would give us some extra voltage for other additional systems in the future (maybe an amplifier for some crazy sound or something). The drawback of that extra 2.4V was that I had to employ a voltage regulator and waste some space to heatsink it, but that didn't seem unreasonable.
The standard Raspberry Pi camera module seemed to be small and light enough to fit the bill, plus I had a few stashed away. There are also relatively easy to use programs (raspystill, raspyvid) that provided a great starting point.
We also eventually decided to add some red LED eyes and use a 10mm white, ultra-bright LED as a headlight so I grabbed some from the parts bin.
Now to drive DC motors in both directions something like an H-bridge would be required. Sure, I could have monkeyed around with some MOSFETs to get one in place, but I decided to go with a little driver board from Pololu instead (uses a Toshiba TB6612FNG dual motor driver). I've used different driver boards of theirs in 3D printing with good results so I figured it would likely save me some headaches.
Since there was no way all of our madness was going to fit into the body from the tank kit it was clear that we were going to have to mount a larger enclosure on the kit's chassis. We selected an appropriately-sized project box from Radio Shack to serve this need. Sure, it would end up looking like a rolling box, but who cares.
The first step was to get the electronics together to prove that all of the ideas would work. After breadboarding everything out I put a board together from prototyping PCB with 8-pin female headers for the motor driver; 4-pin headers for ground, 7.4V and 5V power rails and the regulator with its associated capacitors/rigamarole.
To get the basic structure in place we started chassis from the tank kit and sawed off the connector pieces that the body attached to. From there we drilled some holes through both the base of the project box and the chassis for mounting hardware and motor wires. Machine screws fastened the bottom of the box to the chassis and long M2.5x20mm-ish screws were inserted to ultimately mount the Pi and the custom board mentioned above.
We mounted the Pi and the custom board on the skinny screws using nylon spacers to keep everything apart. Everything was (and still is) wired up using jumper wires. 5V goes to the Pi's 5V in, the Pi's GPIO pins go to the motor driver's inputs and pretty much everything works as expected.
After toggling GPIO pins from test code to move the motors it was clear that we were on the right track. But, you know, the boy wanted more. He wanted LED eyes. Feature-creep happens at home too, I guess. It seemed that if we were going to go as far as to add ornamental LEDs a headlamp would be a useful addition. Since the Pi's GPIO pins don't source enough to get the 10mm ultra-bright LED to burn your retinas I put a board together with PNP transistors to trip it.
With the connection of the camera and mounting of the wheels/tracks we were in good shape. Everything worked with some test code so it was time to turn my attention to putting proper control software together.
Now we came to the point where I'm actually somewhat professionally qualified (I'm a software developer, I don't hack apart toys by trade). It was time to develop the control software.
Since we were sticking with a vanilla Raspbian install on the Pi our options were open. It seemed the path of least resistance was to hack in python and use the "RPi.GPIO" package to control the peripherals. As I stated initially we wanted the system to be web-based. Our needs were simple and small so I chose CherryPy for the job.
To start we kept the video simple. Rather than shoot and stream video we just used raspystill to shoot stills a few times per second. Our web interface would then periodically refresh to get a current view of the situation. If this proves cumbersome we may instead choose to stream video.
In time I'll get the software more organized and get it up on Github, but I'm just not there yet.
Well, here it is in its current form doing my bidding.
Fri Mar 07 2014 00:00:00 GMT+0000 (UTC)
One interesting omission from Amazon's Linux AMI is SElinux and I recently had occasion to install it on a few EC2 instances. The process of installing and enabling SELinux in this environment is actually quite strait-forward, although it can require digging through quite a bit of incorrect and obsolete documentation.
The instructions below are what worked for me using the 2012.09 relase of the AMI. 2012.09 ships with kernel (3.2.30-49.59.amzn1.x86_64), but these instruction will indeed upgrade it.
The first step is to install the following packages which include SELinux and some accompanying tools.
[root@EC2]# yum install libselinux libselinux-utils libselinux-utils selinux-policy-minimum selinux-policy-mls selinux-policy-targeted policycoreutils
Now we have to tell the kernel to enable SELinux on boot. Append the following to the kernel line in your /etc/grub.conf for your current kernel. Note that if you want to boot into permissive mode replace enforcing=1 with permissive=1.
selinux=1 security=selinux enforcing=1
In my case the resulting /etc/grub.conf looked like:
# created by imagebuilder default=0 timeout=1 hiddenmenu title Amazon Linux 2012.09 (3.2.30-49.59.amzn1.x86_64) root (hd0) kernel /boot/vmlinuz-3.2.30-49.59.amzn1.x86_64 root=LABEL=/ console=hvc0 selinux=1 security=selinux enforcing=1 initrd /boot/initramfs-3.2.30-49.59.amzn1.x86_64.img
Now install a new kernel and build a new RAM disk. Don't worry, the options you added above will propogate to the new kernel.
[root@EC2]# yum -y update
Relabel the root filesystem
[root@EC2]# touch /.autorelabel
Now examine /etc/selinux/config and ensure the enforcement level and policy you desire are enabled. In my case I stuck with the default fully enforced targeted policy.
# This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=enforcing # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted
Now reboot the instance
Because the root file-system was set to be relabeled rebooting will take a few minutes longer than usual.
Once the instance comes back up log in and verify your work. If everything went as planned the getenforce command will generate the following (for full enforcement).
[root@EC2]# getenforce Enforcing
And you're done! SELinux is installed and operating on your instance.
Fri Dec 14 2012 00:00:00 GMT+0000 (UTC)
At the office a while back I was experimenting with techniques to initialize MySQL replication for both InnoDB and MyISAM tables without significant downtime. The idea of locking all tables and performing a backup to ship to the slave simply takes far too long. The method that I ultimately ended up adding to my toolbelt was an adaptation of the process outlined in this article by Badan Sergiu which uses a tool from Idera called R1Soft Hot Copy for Linux.
R1Soft Hot Copy
What differentiates this process from a more standard approach is the employment of R1Soft Hot Copy. R1Soft Hot Copy is a tool that facilities the creation a snapshot of a block device. When changes to the original device occur only the differences are placed in the snapshot in a Copy-on-Write fashion (similar to VSS in Microsoft Windows). This allows an administrator to create a functional, mountable backup of an entire device almost instantly with very little effort.
Motivation and Caveats
I'm posting these instructions because I'd like some feedback not only on my adaptation, but also on the initial method. Feel free to use any of this information, but please be careful. It worked for me, but I'm not qualified to write authoritative tutorials on the subject.
Prerequisites and Requirements
I'm going to make the assumption that the reader knows how to setup MySQL replication using the methods outlined in the official documentation and that they've read the source article mentioned above.
Also keep in mind that R1Soft Hot Copy is a Linux utility making this article not directly applicable to other operating systems.
A central theme in Badan Sergiu's article was to avoid locking tables (or using lvm). The cost of not locking tables was a restart of the MySQL service itself on the master; meaning that even read queries were not able to be processed momentarily. My idea was to instead flush and lock tables in the standard fashion while creating the Hot Copy mount. That should allow read queries to still be processed and connection attempts to succeed. Writes will be temporarily blocked, but only briefly and clients should have an error free, albeit slower, experience.
Step 1: Install R1Soft Hot Copy
Use the instructions on Idera's website to install Hot Copy and then run
# hcp-setup --get-module
on the master.
Step 2: Configure master
Enable binary logging on the master server and configure a server id in my.cnf.
On the master create a user specifically to be used for replication.
mysql> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'SLAVE_IP_OR_HOSTNAME' IDENTIFIED BY 'slavepass';
Step 3: Create/mount a snapshot
Ensure mysql has flushed all data to disk and then lock tables so no writes can occurr.
mysql> FLUSH TABLES WITH READ LOCK;
Obtain log coordinates. Record the values of the File and Position fields.
mysql> SHOW MASTER STATUS; +------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +------------------+----------+--------------+------------------+ | mysql-bin.000002 | 1234 | | | +------------------+----------+--------------+------------------+
Create and mount the snapshot on the master. Because all tables are locked the coordinates obtained above will be consistent with the data in the snapshot.
# hcp -o /dev/sda2
... where /dev/sda2 is the device containing the filesystem which houses the MySQL databases to be replicated. Watch the output for the resulting mount point. This process should take mere seconds.
Release locks on the tables. This will return operation on the master to normal.
mysql> UNLOCK TABLES;
Step 4: Shutdown the slave's mysqld and copy the data
Run these commands on the slave:
# /etc/init.d/mysql stop # rm -rf /var/lib/mysql # rsync -avz root@MASTER_IP_OR_HOST:/var/hotcopy/sda2_hcp1/lib/mysql /var/lib/
... where /var/lib/mysql is an example path to MySQL's data.
Step 5: Unmount the snapshot on the master
# hcp -r /dev/hcp1
Step 6: Configure the slave's identity and start MySQL
Edit /etc/mysql/my.cnf on the slave and set a server id.
# /etc/init.d/mysql start
Step 7: Configure and start slave
Now it's time to point the slave at the master and start replication. The MASTER_LOG_FILE and MASTER_LOG_POS should be set to the File and Position fields recorded in Step 3.
mysql> CHANGE MASTER TO -> MASTER_HOST='MASTER_IP_OR_HOST', -> MASTER_USER='repl', -> MASTER_PASSWORD='slavepass', -> MASTER_LOG_FILE='mysql-bin.000002', -> MASTER_LOG_POS=1234; mysql> START SLAVE;
At this point replication should be running and the only major service interruption was that writes were blocked for a short period on the master.
There's nothing fundamentally different in the finished product between replication setup in this fashion and a more typical dump-and-copy process. That means monitoring and maintenance should be quite standard.
Also, thanks Badan Sergiu for posting the original article. It helped me immensely.
Sun Nov 18 2012 00:00:00 GMT+0000 (UTC)
Nubimus123: Hey, hey!
Pr0gram4tez: Ah, dear Nubimus. What inspires this chat window so close to happy hour on a Friday? Surely not a bug wreaking havoc on your users or offensive performance bottlenecks?
Nubimus123: No, Programates. Quite the opposite, in fact. I was interrupting your work to invite you for a beer to celebrate my team's recent success. Surely you will join me.
Pr0gram4tez: Perhaps, my friend. First tell me about your success. I wish to learn from your work while my head is still clear. My week was full of disappointment and your youthful wisdom and inspiration may be necessary for solutions in the week that follows.
Nubimus123: Gladly, Programetes. The victory was one of cryptography. Our client's data contains highly sensitive personal information and credit card numbers. They sought us out to secure their data that it never be consumed by either hacker or fool.
Pr0gram4tez: By Schneier! The protection of data of that nature is indeed important and critical to the order of the state.
Nubimus123: And that is why they rightly entrusted us with securing it.
Pr0gram4tez: Tell me, fellow developer, how did you achieve the security necessary to meet the requirements of your customer. Did you labor diligently designing an algorithm channelling all of your inventiveness? Did you employ all of your training and education designing a computation that was quick to perform while at the same time married to the highest standard of protection? Did you seek the cutting edge of mathematics and piety? My excitement implores an explanation!
Nubimus123: LOL!!!!!111 I must confess to you, who have always been honest with me. It all boils down to how we got the contract initially. Our bid was low because we allowed the gods to do the work for us. We used the cryto API in the framework crafted by the Olympians themselves.
Pr0gram4tez: How wonderful! It is said that mortals achieve their highest greatness when letting the gods handle the lofty computation so that they may focus their efforts on terrestrial business logic.
Pr0gram4tez: Tell me, friend Nubimus, what class of divinely-crafted algorithms did you chose? Perhaps a symmetric-key cipher with Hermetic dispatch? Of either the stream or block subtype? Maybe even an asymmetric-key cipher with Alice and Bob unaware of each other's cryptographic secrets?
Nubimus123: SMH... I believe you've failed to learn the very lesson you described mere sentences ago, Programates. Such details are of the divine and, while important, could only be made poorly by simple, mortal software developers. Though I believe the default algorithm for the framework was something called Athena’s Encryption Standard or AES. I'm lead to believe it's the finest cryptographic work produced by god or man. I know not of style or tactics.
Pr0gram4tez: Interesting. The framework is so robust that the implementation required virtually no attention. You've not studied the algorithm itself?
Nubimus123: Like I said. That's a matter for the gods.
Pr0gram4tez: We can agree that the code behind the cryptography is hallowed, but I'm not aware of methods that tailor the implementation to your business case as a matter of course in any framework. There must be much to the implementation, surely. Can you not harm your client if your implementation is incorrect? Could not the gods themselves attempt to injure your business interests?
Nubimus123: Assuredly not, Programetes. The framework's code is not only holy but was developed in pairs and certified by a consortium of deities eliminating the possibility of any one Olympian's mischief or mistake.
Pr0gram4tez: The first question remains unanswered, methinks. It is not possible for you to hinder the service with even the most well-intentioned practical application?
Nubimus123: Perhaps a layman. We are a professional team. We may not be gods, but I'm confident we consumed the framework as intended. Besides, the gods will assist us. We’ve prayed to them in social media. We’ve sacrificed to them with licensing fees.
Pr0gram4tez: The admission of the potential fallibility of an implementation using the framework troubles me, even if by a neophyte. I can accept the quality of the framework as it stands but you've only offered the credentials of your team as proof the implementation is correct. You say you're professionals, but admit that you accepted the algorithm as a default without significant technical consideration. Did you investigate other algorithms offered by the framework? They’re equally divine and perhaps better suited to your toil.
Nubimus123: No, old friend. We have to deliver! Besides, the customer will be satisfied with reasonable security. What they're really paying for is our work-flow and interface. There's no way parameters to a cryptographic problem will significantly impact the effective security of our application in a negative way within the realm of practicality.
Pr0gram4tez: Whether implemented by either god or man are the cryptographic parameters not identically effective or ineffective? Speed and stability may be the domain of the gods, but would a small key size make you equally vulnerable regardless of the algorithm's origin? Can you not impart your human fallibility on these holy tools?
Nubimus123: I must admit, Programetes, I'm unaware of the practical impact of key size in our implementation, but it's certainly adjustable. It's a problem easily solved with configuration. It's still overwhelmingly in the hands of the gods who sold us the framework.
Pr0gram4tez: Very well, Nubimus. But keep in mind even with adequately-sized keys there are more cryptographic parameters such as the mode of operation. Even if adjustable at deploy-time are you aware of them now? Can you guarantee the data provided by your client will be ciphered optimally upon delivery so you don't have to trouble them for migrations later?
Nubimus123: Another admission, oh thorough Promgrametes, follows. I have no answer as I've not studied modes of operation, but can note it for consideration with the team next week. Again, likely a matter of configration. You've proven the need for an audit so our configuration is correct upon delivery, but nothing has been stated yet indicating code needs changed or that our choice was flawed.
Pr0gram4tez: Please understand my goal is not to convince you to change your code, dear Nubimus! I'm simply trying to understand how productive the framework your team uses is. Perhaps I've turned up some points that require attention as is common for a grey-beard like myself. I do have some questions that are somewhat higher-level if you have time before we leave.
Nubimus123: My thirst grows, and my brain becomes weary on this Friday afternoon, but in exchange for the service you've provided I will certainly entertain them.
Pr0gram4tez: My gratitude is immense, Nubimus. Well then, the keys... Does the framework you employ provide a secure key storage facility?
Nubimus123: It does indeed, but we don't use it. In effort to centralize our data we store the keys directly in the application's relational store.
Pr0gram4tez: I'm puzzled, Nubimus. You store the ciphertext in the relational store, correct?
Nubimus123: That is true, Progametes.
Pr0gram4tez: And, as you've stated, the keys are in the relational store. All of them?
Nubimus123: Quite true.
Pr0gram4tez: So in the event of a compromise, beg the gods not, an adversary, perhaps an agent of Syracuse, would have all the tools necessary to recover the plaintext and use it against you or your clients or Athens itself?
Nubimus123: Well, I suppose so, Programetes. That seems unlikely, though. The attacker would have to compromise multiple, higher-level layers to achieve access.
Pr0gram4tez: What evidence do you have that any other layer of your application isn't equally vulnerable? Have you not used the same strategy of trusting the framework above the judgement of mortal programmers? Is not the desired result of cryptography to protect your clients data in the event the higher levels of the application are compromised?
Nubimus123: Well, I fear that may be true, rare friend.
Pr0gram4tez: Tell me, then. Is the cryptography not potentially rendered irrelevant by your key storage practices. Is the net result that there is little security benefit but there is complexity incurred in the application?
Nubimus123: I admit that my understanding, and perhaps that of the whole team, of how to implement a cryptographic system was inadequate. We became emboldened by the code crafted by the gods and it made us feel invincible. I now know that a tool of the gods in the hands of man doesn't make that same man godlike.
Pr0gram4tez: I regret that our conversation may leave you with your mood diminished, Nubimus. I am sorry for that. It was not my intention to play your adversary, but I am thankful that you and your clients may benefit.
Nubimus123: And I thank you for it, teacher Programetes. Regardless, my eyes can focus on pixels no longer. It's time to close this chat window.
Pr0gram4tez: Beer, then?
Nubimus123: I've changed my mind, Programetes. I feel I must stay sharp for a busy weekend of study.
Sat Mar 10 2012 11:50:32 GMT+0000 (UTC)