Tag: nas

Automatic Shutdown/Wake Up on FreeNAS

In an attempt to conserve as much electricity as I can, I used some python scripting to automatically turn my FreeNAS server on and off as required. Requires programming knowledge.

WARNING: THIS POST ASSUMES SOME PROGRAMMING KNOWLEDGE, as you will have to tweak it to your own setup. If you aren’t able to generally figure out what is going on in this post, you shouldn’t try it. I take no responsibility to damage caused to your system! I wrote this post in a rush, so I may miss out basic steps.

In an attempt to conserve as much electricity as I can, I turn off my FreeNAS server whenever it’s not being used. Unfortunately, this was quite a tedious task to do manually. So, I used some scripting to automatically turn it on and off. The scripting for auto shutdown is done on the FreeNAS server, while auto wake up is done client-side.

Auto Shutdown

Using a cronjob, the FreeNAS server monitors a preset list of computers (via IP addresses). Once all the computers are off (i.e. no longer reachable on the network), it’ll turn itself off.

1) Create a file named shutdown.py with the following code: shutdown.py

2) Edit the IP addresses section with whichever computers you want to monitor in your network, e.g.:

###### IP addresses #####
# IP addresses go underneath this line, one on each line in the format: ip_list.append('x.x.x.x')
ip_list.append('192.168.1.2')
ip_list.append('192.168.1.3')
###### End IP addresses

3) Place the file somewhere on your server, and make sure it’s executable. You can make a file executable by SSH’ing into the FreeNAS server, and entering the command chmod +x /mnt/path/to/shutdown.py, changing /mnt/path/to/ to the directory that shutdown.py is in.

4) Create a cronjob to run the file. You can do this via the Web UI. For example, the following settings run the script every 5 minutes from 12am-5am. You can adjust this to your own liking.

Automatically Wake Up

Whenever any of the computers in the house are turned on, they send a Wake-on-LAN signal to the FreeNAS server and automatically mount the shares.

Mac OS

Python is required to run this script! Basically, these scripts will check every 3 minutes that the FreeNAS server is on, and that the network shares are mounted. If not, it will attempt to turn on the FreeNAS server (by sending WakeOnLAN) and mounting afp shares. Please feel free to edit the scripts as you see fit, or change the directories they are in, of course making the necessary changes.

1) Create the file com.hoongern.nasmounter.plist in /Users/<your username>/Library/LaunchAgents

2) Create the file nas.py in /Library/Scripts/. YOU HAVE TO EDIT the configuration in the script as required:

3) Restart your Mac. If I remember, launchctl should find the new task. You can run launchctl list and check if com.hoongern.nasmounter is listed. If it’s not, I can’t actually remember what to do. I don’t actually use Mac OS, you see…

Windows

To be honest, I haven’t written a script for Windows. Not that it should be hard at all – just use a similar python script, the Task Scheduler, and the “net use” command to mount Samba shares. If there’s enough interest, I can write it up and post it here – let me know!

Linux

Again, no script yet. Shouldn’t be difficult with a similar python script, cronjob, and the “mount” command to mount Samba/NFS shares.

FreeNAS: Simultaneous AFP/CIFS shares done neatly

Learn how to hide Mac OS specific files from being displayed to a Windows client when using simultaneous AFP and Samba/CIFS shares.

In many networks these days, you’ll probably have more than just Windows or Mac OS clients. If you regularly exchange data between Mac OS and Windows/Linux using a flash drive and have hidden files enabled, you may notice a bunch of .DS_Store folders and other various files beginning with a dot. Personally, I find it somewhat annoying that Mac OS litters whole file systems with these files (In the same way, I also hate those thumbs.db files Windows generates)

In a unified file server which serves multiple operating systems, we ideally want each client to have a good experience browsing for files. We don’t want Windows users to be bogged down in a mess of files they have no clue about, and we don’t want them accidentally deleting files which may be important to the Mac OS experience.

There are no adjustments which need to be made on your AFP shares, since only Mac OS clients access those shares and know what to do with these files. However, these files should be hidden on CIFS shares. Here’s how you do it in FreeNAS 8 (The solution applies to anyone using CIFS, not just in FreeNAS):

1. Navigate to your CIFS share and press edit

2. Find “Auxilary Parameters at the bottom”

3. Enter the following text in the box

veto files = /Temporary Items/.DS_Store/.AppleDB/.TemporaryItems/.AppleDouble/.bin/.AppleDesktop/Network Trash Folder/.Spotlight/.Trashes/.fseventd/
delete veto files = yes
hide dot files = yes

Vetoing a file will render it completely invisible to anyone accessing the share. It differs from hidden files in that hidden files are hidden on the client side, but vetoed files are hidden on the server side even before the list of files is sent to the client.

The first line veto files specifies a list of file names to veto, each entry separated by a ‘/’, and you can also use wildcards (‘*’ and ‘?’) to specify multiple files with a single pattern.

“delete veto files” allows CIFS to delete any vetoed files within a directory when that said directory is deleted. If this is not set to ‘yes’, deleting a directory could fail.

“hide dot files” is optional, but simply sets all files beginning with a dot (which are hidden files in Mac OS and Linux) with a hidden flag.

4. Restart CIFS from your Services tab

If all is well, browsing your shares from all computers should now be a pleasant experience!

If you’re ever in doubt, or if you want to find out what other options can be entered, do check out the smb.conf documentation.

Network File Server using FreeNAS (Part 1)

Juggling around files can be a nightmare when you have multiple computers, especially when you use multiple operating systems. Trying to back up files can easily turn into a mess once you have multiple copies of files here and there, and you may constantly worry about hard drive failure.

By building a NAS yourself, you can solve these issues and save a lot of money compared to buying a commercial NAS product!

Juggling around files can be a nightmare when you have multiple computers, especially when you use multiple operating systems. Trying to back up files can easily turn into a mess once you have multiple copies of files here and there, and you may constantly worry about hard drive failure.

These are some of the reason which drove me to set up a NAS (Network Attached Storage) server to serve my files across my network. My requirements were the following:

  • At least 2TB of storage
  • Able to serve Windows, Mac OS, and Linux (SMB, AFP, NFS) at gigabit speeds
  • Low power consumption for 24×7 usage
  • Protected against hard drive failure
  • Protected against accidental file deletion
  • As cheap as possible

There are various companies which provide NAS solutions – for example, Synology and QNAP, as well as regular hard drive / network device manufacturers like Western Digital and Buffalo. However, as I looked through their products, I realized that while they looked very nice, they tended to be expensive and limited (depending on how deep your pockets are). For example, the Synology DS411J and QNAP TS-410 (both 4-bays) both cost around USD360, and once you’ve got them, they don’t tend to be that flexible.

So, I decided to DIY a server. Firstly, I looked at OS solutions, and settled on FreeNAS 8 (Using something like Solaris/FreeBSD would be even more flexible, but for simplicity, I went for FreeNAS which is small and has a decent web interface). The main attraction I was looking for was ZFS, a file system built to ensure data integrity. It can be thought as a marriage between hardware & software RAID, which is very beneficial because it avoids many of the write hole problems with a RAID system, and can self-heal your data. I won’t go into the details of ZFS, which can be found elsewhere, but it is clearly a very capable file system.

ZFS would provide me the following:

  • RAID-Z1: By using one of the drives in a pool for parity, I would be safe from a single hard drive failure.
  • Dataset quotas: Ability to limit datasets (or to effect, the shares) to a certain size.
  • Snapshots: You can think of this like Time Machine for Mac OS, although, to me, even better, because snapshots are instantaneous, and provide me safety against accidental file deletion, etc. They don’t take up any extra space, and allow me to roll back my file server to a particular state. Here’s how to use snapshots with Windows & Mac OS [Coming soon]
  • On-the-fly compression: Using fast compression algorithms (You can actually choose from a range – Gzip/LZJB), data can be compressed when written to the hard drive, saving space and in some cases, enabling even faster speeds on the hard drive. This is because most reading/writing is limited by hard drive speed and not CPU compression speed (At least with a modern CPU).

And having a DIY NAS server would give me the following:

  • A fast CPU for on-the-fly compression/transcoding – which I can upgrade in the future if needed
  • Flexible amount of RAM
  • Up to 6x SATA HDDs, plus with PCI-E expansion, I could easily put up to 15 Hard drives in total, for a LOT less cost than a commercial NAS
  • Ability to upgrade to 2x1Gbps network interfaces (or more), to double my network bandwidth
  • USB3.0 support

Of course, a DIY solution would tend to consume a bit more power than a NAS solution, given that it has a lot more CPU/RAM, and a lot depends on the power supply efficiency.

Parts & Build

In the end, I decided to go with the following:

  • Intel G620 2.6GHz Processor: The cheapest Socket 1155 CPU I could find at the time. It’s already a lot faster than needed, although if you do use on-the-fly encryption or compression, you can saturate it
  • Intel DH67BL-B3 microATX Motherboard: One of the cheapest boards I could find with at least 5x SATA ports. I decided to go with this because it had an Intel network controller, and according to some reviews I read, the lowest power consumption. It also gives me USB3.0 support which could be useful in the future.
  • 2x4GB Kingston DDR3-1333MHz RAM: Since RAM is cheap, and ZFS benefits from higher levels of RAM, 8GB made sense. The motherboard, supporting up to 32GB of RAM, can be easily upgraded in the future
  • 3x2TB Western Digital WD20EARX “EcoGreen” Hard drives: Not nearly the fastest drives around, but from tests (at least of the earlier generation WD20EARS), they appear to have the lowest power consumption
  • Corsair 4GB Flash Voyager USB flashdrive: One I had lying around to install FreeNAS on

All this added up to a grand total of USD422, hard drives included (USD196 without). I didn’t have a free case lying around at the time, so I just built it open:

After updating the BIOS, adjusting fan speeds, etc., I installed FreeNAS 8.0.1-RC1 onto my flash drive and booted up. Everything was detected without problems and the web interface was accessible. I created a RAIDZ1 array using the 3 drives, resulting in 3.56TB of usable space.

(Note: I had to use the wdidle3 tool to ensure my drives wouldn’t load/unload their heads so often, which could lead to premature failure)

Performance

Using the linux dd tool, I did a quick benchmark of the hard drives, which gave 217.7MB/s writing and 226.1MB/s reading. More than fast enough for me, given that gigabit LAN maxes out well before that anyway.

Next, I tested the network performance using iperf, and with 2 clients connected, the interface managed to push 950Mbps (119MB/s). All was looking well, so I quickly set up a CIFS/SMB (Windows) share and copied a file over. Both ways, I was getting 112MB/s. I also set up an AFP (Apple) share which got me the same 112MB/s.

Conclusion

I managed to get a file server which has great data integrity features, and can serve all my network clients. In addition to the regular network shares, I also use it as a “Time Capsule” of sorts for the two Mac OS machines on my network. In particular, for much less than what it would cost to buy an equivalent NAS or Time Capsule!

FreeNAS is still a work in progress, and isn’t perfect right now (still waiting for them to add VPN and torrent support), but it’s definitely on its way.

Further

FreeNAS: Simultaneous AFP/CIFS shares done neatly