Playable Lego Piano

Made with ♥ by DebSource Code

Making the Lego Ideas Grand Piano fully playable using a Raspberry Pi, and using it as a game controller.

This is a modification of the original set so that each individual key is recognized. This makes it fully playable, unlike the original. The feel of the keys is unchanged and it supports full 25-key rollover with realistic sounds for each key, thanks to samples that my friend Mihir provided.

Code and diagrams are included for anyone who wants to try modifying their own set. The modification was designed to be easily put together using cheap and easily available materials. If you run into issues, you can open a pull request or contact me through GitHub and I may be able to help (no guarantees).

This is documentation for what is the second major version of this project. It was designed to be far more reliable and easy to reproduce than the first. I wouldn't have recommend for most people who own the piano to try the first version of the project, but with some time, I think anyone could reproduce this one quite well.

Demos

Realistic Playability

The cool part is that the software supports full 25 key rollover, so you can press multiple keys at the same time and the sound will still be realistic.

Some keys are a little uneven since I was experimenting with switch types, but if you want to make your own, using the same switch creation procedure described further down this page should result in pretty even keys, as long as you're careful and test evenness as you go.

Flexible, Open Source UI

The code for the user interface is served by the Pi itself. It's a web UI that should be accessible over the local network using any device with a modern web browser. It's fully open source and not dependent on an app store (or even the internet!), so it doesn't carry the same risks pertaining to being discountinued, etc.

Gaming, for some reason

You can also use the piano as a very strange game controller.

I tested it with Batman: Arkham Knight, since the combat doesn't require many potentially uncomfortable button combos. I played a single player game because the piano didn't play well with anticheat in some multiplayer titles. The latency is not bad, and using UDP made it very easy to program. Theoretically, one could improve this by reprogramming the protocol in bluetooth, but if you're worried about latency, (I hope) you wouldn't use this in the first place.

You can play wirelessly, but it was plugged in when I recorded the video since I had been playing for a bit and needed to charge the piano.

Note that I haven't updated the code for the new switch design, but you can dig through the git history or get in touch with me if for some cursed reason you want to reproduce this.

Making Your Own

Disclaimer: A little technical knowledge is required when putting this together - you'll need to know the basics of how a Raspberry Pi works, and how to run some simple code. A little Googling goes a long way here - these are things that should be doable to figure out for someone who was able to put together the base set. The procedure should be fairly reproducible with some tinkering. That being said, I make no guarantees about your success - the result has worked pretty well for me, but that's just a sample size of 1. I recommend reading through the information in its entirety before you start making your own. Please let me know if it worked!

What You'll Need

  • Lego Ideas Grand Piano, Set #21323
  • Raspberry Pi - Get a standard sized Raspberry Pi, and make sure that the model you get has a wireless adapter (wifi). Other than that, the exact model shouldn't matter
  • Accompanying Raspbrry Pi stuff - Make sure you have a keyboard, mouse, monitor, correct display cable, micro SD card, etc. so you can set up the Pi. I'd recommend getting a longish power cable, unless you're going with the power bank backed piano.
  • Raspberry Pi jumper wires - Make sure at least one side of each wire has a female connector so you can connect it to the Pi. We'll be removing the Dupont connector thingy at the other end. We'll need 26, assuming none break, so get extras.
  • Copper/Aluminium Tape - Important Note - Make sure you buy tape where the adhesive side is also conductive.
  • Small Paper Clips - Important Note - Make sure their widths are less than the width of the thin part of each key (one unit) and that their surfaces are conductive. We'll need 25, assuming none break, so get extras.
  • Power Bank - Optional - Only required if you want your piano to be wireless.
  • Scissors
  • Tweezers - Optional - You can make do without these, but they make things much easier, and also help remove tape if you're undoing mistakes.
  • Continuity Tester - Optional - Highly recommended so you can check that circuits are closing as expected while you're building the switches. Make sure you get one that can check circuits that aren't live. Mine was small and quite cheap.
  • Rubbing Alcohool/Alcohol Wipes - Optional - You'll only needs these to remove tape residue if you mess something up.
  • A Wifi Network - Optional - May not really be required, but highly recommended, at least to set up. See the software section for a possible alternative, where the Pi hosts its own network.

The Software

You can find the code here.

Setup Process (do this before starting hardware work)
The setup process has been greatly simplified since the first version. A lot of items here are not in as much detail as they could be, since online tutorials for these steps online will be more comprehensive, and if something doesn't work right away, Google is your best bet anyway. For each step, you'll very likely have to just Google how to do it, but let me know if something looks off.

  1. Set up the Pi with an OS, if it doesn't have one already. I believe I used the default Raspbian.
  2. Download the Source Code, either by cloning it using git or downloading and unzipping the repo somewhere. From now on, you'll be working in the `code` folder.
  3. Install the requirements from the requirements.txt file using pip.
  4. Run `piano-flask-server.py` using Python - I believe you'll need to be using Python 3, and the pip you used will have to match.
  5. Connect your Raspberry pi to your wifi network, and set it up so that it will only boot when a wifi connection is available (I believe you can do this from the BIOS). Alternatively, you can look up how to get the Pi to host its own wifi network, but I haven't tested this.
  6. Find the local ip address of the Pi - using `ifconfig` should do the trick.
  7. Navigate to `(the pi's ip address):8080` (ex: `192.168.0.13:8080`) in a pc or phone browser that's on the same network, as the Pi, and make sure the UI shows up. If it doesn't, check that everything is connected to the network correctly.
  8. Set up the Pi so that the Python script runs at startup - I believe modifying my bashrc did the trick.
  9. If everything went well, you shouldn't need to change software on the Pi anymore, so you can run it headless from now on.

Room for improvement
The software is far from perfect, and could be improved in many ways, most of which would offer much better responsiveness. The reason these haven't been done is because I haven't yet been able to justify the time commitment, but let me know if you'd be interested.

  • Switching to web Bluetooth - Assuming the APIs are mature enough.
  • Switching to app based Bluetooth - Annoying due to being tied to app stores.
  • Switching to web sockets in browser - Getting this to work over local network seems painful.
  • Adding support for the pressing of the pedal being recognized, and incorporated into the sound.
  • Porting the old code over - these are non browser based solutions, which I don't recommend starting with because of the increased setup. That being said, if these were updated for the new switch designs, the responsiveness would be much better (these are UDP based) and it would allow for functionality such as gaming or adding a speaker to the Pi itself.

The Hardware

The key idea behind the design is using copper (or aluminium) tape around the base and each key and connecting each key's tape to a GPIO output on the Raspbrry Pi to determine whether or not a key is pressed by checking whether or not the circuit was closed. This design results in keys which feel unchanged from those the original set, and hardware that can be built without needing to get a bunch of switches or printing a circuit board. In these pictures, I used a mix of aluminium and copper tape, but practically, it doesn't matter. If your tape is thin, it might tear easily - an easy fix for this is just to double up.

We start by building just the keyboard portion of the Lego set (this starts at bag 11 of the instructions) - we'll revisit the rest of the set once this is working reliably. If your piano is already built, take it apart and lift out the keyboard and power functions assemblies so that you can work on the areas in the following pictures - it's annoying, but not as tricky as it looks, based on my experience. Once that's done, we'll wrap the entire base underneath the keys in tape (or the individual plates which are then linked) - this will be our ground once we connect it to the Raspberry Pi's ground. To do this, connect it using the tape across the bottom of the base so you can connect to the Raspberry Pi ground wire on the other side (remove the Dupont connector using the tweezers, and wrap the tape around the end of the wire). We also add a wire on 25 separate GPIO pins starting from 2, and an additional wire for ground (consult a pin map of your model for this step). We haven't yet built the switches, but this is a good time to test out the software by manually closing the circuits and making sure the keys on the UI light up/produce sound.

Take apart the bottom of a key (the three pieces under the long brick)

Slightly pry open a paperclip, so that it can lay within the 3x1 brick. Seal it in with the 3x1 plate between the sides of the paperclip

Cut long strips of tape that are just a little less than the width of the thin part of each key. Lay them along the length of each key so that they lay over and secure the exposed part of the paperclip (eyeballing the length usually works, and you can fix this with more tape anyway).

Use the cone that the key rests on to poke a hole in the tape where it would normally go. Cut a thinner strip of tape along the side to reseal this, and check that the connection still works.

For the black keys, the tape should go over the edge slightly, but it can stop a bit before the edge for the white keys. The only important thing is that they connect to the base when pressed down, so check the connections.

Remove the Dupont connectors from the ends of the wires using tweezers, and bend them around the ends of each paperclip (be careful - use the tweezers for this). Secure the connection with the tape. Repeat the process with each key in order. Try not to mess up the key order, but if you do, the software can allow you to remap the keys (as long as you have connected the wires to GPIO).

Don't forget to connect the ground wire to the base

Once again, it's a good idea to check the connections before proceeding.

Now we get back to building the piano. Ignore the transparent tape on top of the keys in these pictures - these pictures were from the old switch design, but the steps from this point on are the same. We're going to build the rest of the Piano around the keys. Starting from the beginning, we omit the portion where we add the bracket for holding the power functions battery box included with the set (since we don't need it), but otherwise proceed as usual until step 76.

The next pictures show the differences in the next few steps. We use the parts from these sets to seal in the wires between the keys and the Raspberry Pi, and try to orient it so that the ports are easily accessible from the right side panel, (and/or add a battery bank here if we want it to be entirely wireless).

Make sure everything works before you seal it in. We then build the rest of the piano as usual. The only other tricky part is step 288, but isn't too bad if the keys are handled a little carefully.

Once we're done, the Raspberry Pi should be accessible from the side panel, so we can plug it in, add a power bank, etc.