Brainiac Storage Server
At a certain point, my storage needs began to exceed that of your average user. The typical mess of external drives just wasn't doing it anymore and most cases that could actually support a large number of drives were absolutely monstrous in size (notably that is no longer true, there are some very nice cases from Corsair and Phoenix that have space for up to ten drives and conform to a more classical build pattern). I've always been a fan of a quote attributed to Hannibal: "I will either find a way or make one." so at some point I decided that I simply needed to build my own server. It's worth pointing out that I eventually found a subreddit /r/datahoarder filled with people who took this to an even greater extreme and build rack servers in their basement as a matter of course. This article is not about the biggest server you can possibly make, but I like to think that it is about one that can provide the greatest utility for the price. The fact of the matter is that even now, three years after I built it, it costs significantly less than a pre-built Synology NAS or Drobo with the same amount of storage space, which has a much weaker processor, no choice of operating system, and lacking in plenty of other customizable features. That being said, this server build also comes with a lot more learning about stuff you didn't even realize existed compared to buying a ready-made Drobo. For me that's a major advantage, but that's largely a matter of personal preference.
I have made several posts to reddit about my build as well as a series of upgrades that I have made to it over the years. The final post that summarizes my current build can be found here and here while the original post I made of the first iteration can be found here but is considerably less detailed.
According to my naming convention of using Sci-Fi robot names, this server was named Brainiac after the Superman villain who attempted to store everything about the world in his brain and destroy the planet in the process.
The Parts & Pricing
I should point out that as I built this server in the summer of 2013, so by the time you read this some of the facts may have changed - prices are likely different, there are more efficient processors on the market, there are almost certainly larger hard drives to be had for the same price, and, sadly, some of the parts may no longer be available. I apologize, unfortunately there is no way that I know of to make this article future proof, but I think it still provides a nice guide for anyone looking to build their own server like this.
| Part | Price | Description |
|---|---|---|
| Case | $200.00 | This is the critical peice of the whole build. The fact of the matter is that cases like this seem to be difficult to come by as few manufacturers actually produce them for any kind of market. Since I built Brainiac, new cases have come out that can be used for a server like this with at least 8 drive bays. The semi-complete list can be viewed here, but it is difficult to find another case that has hot-swappable drives without going into the actual rackmounted cases. This case is tight on space and pretty much everything has to be measured carefully to be sure that it fits inside, but the result is that the server does not take up a lot of space. The case admittedly looks a little shoddy, but I can confirm that I have had absolutely no problems with it in terms of anything breaking inside. I suppose it goes without saying that you should probably avoid dropping a server filled with hard drives. I should also note that while there is not a lot of space inside the case, there is an area where the motherboard would go that ends up having some room as well as a relatively free space above the drive-housing that should give you plenty of space for wire management. |
| Motherboard | $110.00 | I decided to get this motherboard mainly because it was the cheapest at the time that was able to fit the processor I wanted (listed below), had wi-fi, and two ethernet ports. I figure that the Wi-Fi is probably unnecessary for people looking to build something like this, but I occasionally require it so I figure its there if I need it and the mobo is really not that expensive. Again, absolutely no complaints, fits into the case just fine. Ultimately, the previous motherboard that I used was the main reason I started upgrading the server because something went wrong with the original and after the Raid Card broke (described below) it would not accept any others, even the same model as I was using before. This one had no such issues. |
| CPU | $130.00 | This CPU was chosen as the result of a bit of research on part of myself and a friend of mine. The idea is that I believe this was the i3 CPU with the lowest power consumption (35W) while at the same time being just powerful enough to do the transcoding for the Plex Server that I run off of this system. The previous CPU would struggle with it a little bit (I often stream to multiple people at the same time), so the decision was made to upgrade to this one. As of right now, after having run on this CPU for several years, I've had no problems transcoding video for up to 8 streams, 4 of which were on the LAN and the rest were to the outside. I've never had more than 8 streams at a time, so I cannot comment beyond that, but for most people reading this, that should be sufficient (obviously taking into account the quality of the video being streamed). I should probably point out that this CPU has apparently increased significantly in price since I purchased it, possibly because it has gone out of production, but I am sure that there is another processor out there with similar power consumption and performance for less. When looking for a CPU for a server like this it is extremely important to note the size of the cooling fan on top of the processor (described below) to make sure that it fits. |
| RAM x2 | $80 | I did not actually buy this RAM, but this is the closest thing to what I used that I could find, I had some lying around from a while back so I saved a bit of money there. The RAM I am using is basically the same. No complaints, has been working fine, but if you find a cheaper DDR3 RAM that you like for your build, then more power to you. |
| SSD | $60.00 | This is actually the first upgrade I made to the server since the original build: One problem that the old server had was that I used the Plex Server software extensively which stores a lot of metadata per every media file placed there (and if you're storing music this phenomenon is exacerbated, many little media files, lots of metadata) and the 32GB that I was using originally simply did not cut it. Notably, this upgrade is not necessary, one could always move the Plex metadata folder to the storage drives and create a symbolic link in its place, but it may have a negative impact of Plex Server performance. As with all the other parts listed here, its probably no longer the best for the money and cheaper, larger SSDs have probably come out since then. So long as they are the same form factor and capacity, they will probably be just fine. |
| RAID Card | $150.00 | For most people looking at this, this is probably the most important bit of info: In my opinion, this is the RAID card to get. I had lots of trouble with literally all of the other ones I've tried. The one I used originally had some sort of issue with the updated Linux Kernel and I had to constantly re-compile and re-install drivers literally every time I got an update. Eventually it died for no apparent reason, and when I tried ordering other cards, literally none of them worked properly and I had to return all of them. It was extremely frustrating and support was often of no help. I can say the following about this card: The way that I use it, it works perfectly fine. I do not actually use it as a RAID card for a hardware RAID array, instead the way I use it, it's more like an eSATA hub allowing me to hook up 8 drives to a very small motherboard. If you simply initialize the drives in the pre-bios menu (described below) individually, they will appear as individual drives when the OS loads and that's all you need if you're using BTRFS to make a software RAID10 array like I do. |
| PSU | $70.00 | Its absolutely critical to get the correct power supply partially because not all PSUs will fit the case. Specifically, the case requires that it be a 1U Power Supply that is less than 230mm in length. This one is just a few millimeters less and I even had to partially disassemble the case to get the cables to fit between the PSU and the wall of the case, so it was definitely a tight fit, but it worked well enough. This was the most powerful PSU considering the price and size. Notably I think that its overpowered for the task but I did not want to risk it failing when it had to spin up all 8 drives. |
| HDDs x8 | $1080.00 | So, judging by everything I read, these are the Hard Drives to get for a NAS. They seem to last very well and some website that I do not remember right now (might link later) has actual failure statistics over time and these guys come in a close second to a much much more expensive 4TB drive. Therefore I strongly recommend these for your NAS. I have not had any issues with them thus far. I have done something I initially feared I would regret with them though. As you may notice, on the website they cost ~$180.00, but at the cost listed here I had to have purchased them at $135.00. This is because I bought refurbished drives. As you can understand I bought these upgrades one by one over a fairly long period of time, and the drives were the last upgrade. The reason I bought all 8 drives was mainly for the purposes of experimenting with RAID configurations and new file systems (knowing what I know now about BTRFS and having epxerimented with it I can say that this is not necessary, but when I was buying them, I was not sure). It so happened that money was very constraining at the time that one of my older drives seemed likely to fail so I had to buy 8 drives or face the prospect of losing data so I bought them refurbished. I figure that if one or two fail, I have older 4TBs that I can use to replace them. As of the time that I am writing this post, I am planning to purchase 8TB drives to slowly increase the capacity of the server. I should note that for me this is made possible with the use of the BTRFS file system and I cannot vouch for the same thing being possible with a different one. |
A note about prices: When I wrote this article, a few years after I built Brainiac, I actually went through and wrote in all the prices as they were at the time and if you look at the linked reddit posts, you will find that they are slightly different from the originals. Perhaps the strangest thing I noticed, however, is that the prices have barely changed at all since I ordered the parts and built the machine. Yeah, of course all the store websites are telling me that I'm saving X% when I buy this particular part, but its all a rather obvious lie as I have a reasonably reliable record of what those prices were three years ago. In the worst cases, the prices actually went up with the CPU costing literally twice as much for some reason and the HDD prices that I would most expect to decrease did not change even a little. This does have one significant benefit besides teaching me to never trust those "on sale" percentages: This server is a good investment in the sense that the build is fairly future clear
The Build
There is not that much that I can say about the build, but that is partially because I've been building computers for so long that there are things that I take for granted. In general the build was not entirely painless, but it was not difficult either. If I had to do it again (and I did, I built three of these in total, one for me and two for my friends) I think I could do the whole thing in under two hours.
I did post several pictures of the building process which you should feel free to check out here and here.
The things that I had trouble with are as follows:
The drives in the images above are different from the ones described in the table because when I first built this server, I was just experimenting with leftover drives lying around.
Space, especially when it comes to the CPU Fan: I should probably note that I am not the one who had this issue, but I felt that it was important to note because someone who made a similar build did have it. Specifically, if you buy a processor other than the one that I have listed above, then it is possible that it will have a more standard Intel CPU fan that comes with it, instead of a special, small form-factor one. It is also possible that since that processor was released, Intel has simply increased the size of even its small fans. This poses a significant problem as normal fans will most certainly not fit in the space between the CPU/Motherboard and the drive housing. If you run into this issue, do not fret, simply purchase this fan which is made specifically to fit into a small space and I can confirm that it does indeed fit into this server.
Wire control: The mess of wires that will inevitably need to fit into this case is an issue. Simply keep this in mind when constructing the server and make sure to have a place to put extra wires that is out of the way from the motherboard. Obviously make sure that the wires are not too close to any of the fans, you do not want those to be touching. There is space between the top of the case and drive housing where most of the wires can go so hopefully it should be OK.
Buying the Drives: As I noted above, if you use BTRFS, it is not necessary to buy drives all at once. You can simply keep adding drives to the server as you need them. Just make sure that you read up on BTRFS and are running btrfs-balance after adding them to the BTRFS array as well as any other necessary commands. The key information about BTRFS is described in the software section.
Setting Up The Drives: One of the things I did not do is set up a hardware RAID array. If that is what you're looking for, I cannot say whether the card above is any good. Forthat matter the opposite seems to be the case as a friend got back to me saying that he was having a lot of trouble making the card work for any kind of hardware RAID array. Instead, in my setup all of the drives are initialized in the RAID BIOS Menu (loads before the Mobo's Menu) as individual JBOD "arrays" (Just a Bunch Of Disks) which is the simplest configuration. This effectively makes the RAID card into an eSATA "hub" and each of the drives appears in the /dev/ directory independently. From here, I used BTRFS to set them up as a Software RAID array (described below).
The Software
OS - Debian Jessie: I was originally running Ubuntu 12.04 on this server mainly because that was the operating system that I knew best, but eventually I installed Debian Jessie which was the Testing edition at the time and has since become the stable edition. I have to say that I am extremely happy with using Debian on this server. I have kept it running for more than a year now without the need to reboot (with the help of a PSU), the drivers for the RAID and all the other components seem to come standard and I did not have to install anything special, and all the required software listed below is well supported. I've always been a big fan of Debian and I think that the stability that comes with it is pretty important for this server build. That does not mean you cannot use other operating systems for this, I know for a fact that Ubuntu will work (because that's what I originally installed), as will CrunchBang (now BunsenLabs), and CentOS. I also believe that any ubuntu derivative should work just fine. Please feel free to make a headless install as its totally unnecessary to ever do anything with this server with a GUI.
File System - BTRFS: This is the real critical component of this system. I don't think I would've been able to use it the way I have been without it. When I finally learned about BTRFS, I was literally blown away by what it can do. I am very happy with it and I strongly recommend using it for a number of reasons. (first off for those looking to learn the basics, I recommend this article). First off, BTRFS is not so much a file system, as it is a set of software encompassing a lot of things that file systems to not normally do, which caused some issues in the linux community as it broke quite a few barriers that encapsulate linux design. Nevertheless, I am very impressed. First off, it has features in place to prevent bit-rot, which you can read about here which also summarizes a lot of information about BTRFS. Secondly, it has a completely built-in software raid that's separate for data and metadata which you can set when you initialize the filesystem. You can arbitrarily add and remove devices while the file-system is running. There are a few caveats to this one, namely that the file-system needs to reconfigure and restructure itself after a device is removed, and you have to work in the constraints of the raid system being used (so you cannot remove more than one disk at a time from a RAID1 system for example), but this is very impressive nonetheless. It can take snapshots and has a ton of snapshotting features that I have not even scratched the surface of yet. Another cool feature that I totally did not even imagine was possible is that you can change the raid configuration basically at will (giving the system enough time to restructure the data). You can literally go from RAID0 to RAID10 with just a command assuming you have enough space. The big conclusion I got from all of this is that using BTRFS you don't even have to buy all the drives at once as I did, you can buy them one by one and just keep adding them and its truly very very easy. I haven't been so pleasantly surprised by a new software like this for a while.
I really cannot praise BTRFS enough after having built this server. I have never encountered a complex program/system like BTRFS that was as easy to use with a few command line incantations. There is very little to learn and the concepts are well encapsulated. I use it with RAID10, but it has several others. As of the time when this article was written, RAID5/6 on BTRFS is inadviseable that part of BTRFS is still very experimental and can fail when trying to recover data.
Local File-Server - Samba & FTP: What can I say here? Nothing sophisticated, everything is simple and clean. Both Samba and FTP allow me to restrict users based on accounts and control their access to specific folders while providing decent transfer rates within the local network. Notably there is no local security or encryption, but I figure that this is not necessary, especially if this thing is behind a router.
Remote Access/File-Server - SFTP: When working remotely of course, encryption is key (pun intended) so there is a port open to the SSH/SFTP server the provides me with all of my remote access needs.
Media Server - Plex Media Server: My personal favorite and arguably the best media server software out there, Plex, when combined with a router and forwarded ports, provides access to media both locally and remotely on all devices. It includes a web interface and mobile apps that anyone can use. Its also really nice that they have Chromecast support because it makes it easy to watch stuff on TV. I again highly recommend Plex for this kind of setup. It should be noted that Plex uses transcoding when streaming media remotely, therefore the processor needs to be powerful enough to handle transcoding/streaming for however many users you plan to support on your server. Paired with a decent internet hookup (and given that comcast doesn't crush net neutrality) I am able to stream all my videos (and I have friends who successfully use it for music) both at home and wherever I might be that has internet access. Makes movie nights very fun and easy, we pick out what we like and simply watch it regardless of what kind of device is available and where we are. Heck my grandmother has started watching movies off of my Plex server in Russian (for that matter it has automated subtitle download functionality which means that I have subtitles for all my movies in both Russian and English).
The Setup
The way that I have it set up right now is that I have it sitting in the basement along with a "compute" server that I called Marvin (yes, after Hitchiker's Guide to the Galaxy) which performs intense computations for my work, hobbies, research, and school assignments, and uses Brainiac as storage. Attached to the same switch in the basement, I have a small 4-drive Seagate NAS. Its really crappy to be honest, but its good enough to run an STFP server. The reason for the NAS is that I am really worried about losing data and never sure of my skills with something like BTRFS. As a result, I felt the need to get a small, pre-build NAS, that serves as a tertiary, loosely coupled backup of all the data on Brainiac. Unlike Brainiac, the NAS is only RAID0 and hence can contain all the data on just 4 drives. It is backed up to manually with a pre-configured RSync alias from Brainiac. The reason for such a setup is that if I ever mess anything up with BTRFS (and I have: for the record, never force shutdown of a computer while it is doing a BTRFS-balance), having RAID10 would do jack-diddly squat to save my data. Therefore, I felt it prudent to have a tertiary backup that is uncoupled from Brainiac and its BTRFS setup, just in case.
Conclusion
So yeah, there it is, I hope you enjoyed my description of how to build and configure a nice, small, personal media storage server, and I hope that you find this post useful. Please feel free to email me if you have any comments or questions.