Tag: networking

Automatic Shutdown/Wake Up on FreeNAS

In an attempt to conserve as much electricity as I can, I used some python scripting to automatically turn my FreeNAS server on and off as required. Requires programming knowledge.

WARNING: THIS POST ASSUMES SOME PROGRAMMING KNOWLEDGE, as you will have to tweak it to your own setup. If you aren’t able to generally figure out what is going on in this post, you shouldn’t try it. I take no responsibility to damage caused to your system! I wrote this post in a rush, so I may miss out basic steps.

In an attempt to conserve as much electricity as I can, I turn off my FreeNAS server whenever it’s not being used. Unfortunately, this was quite a tedious task to do manually. So, I used some scripting to automatically turn it on and off. The scripting for auto shutdown is done on the FreeNAS server, while auto wake up is done client-side.

Auto Shutdown

Using a cronjob, the FreeNAS server monitors a preset list of computers (via IP addresses). Once all the computers are off (i.e. no longer reachable on the network), it’ll turn itself off.

1) Create a file named shutdown.py with the following code: shutdown.py

2) Edit the IP addresses section with whichever computers you want to monitor in your network, e.g.:

###### IP addresses #####
# IP addresses go underneath this line, one on each line in the format: ip_list.append('x.x.x.x')
ip_list.append('192.168.1.2')
ip_list.append('192.168.1.3')
###### End IP addresses

3) Place the file somewhere on your server, and make sure it’s executable. You can make a file executable by SSH’ing into the FreeNAS server, and entering the command chmod +x /mnt/path/to/shutdown.py, changing /mnt/path/to/ to the directory that shutdown.py is in.

4) Create a cronjob to run the file. You can do this via the Web UI. For example, the following settings run the script every 5 minutes from 12am-5am. You can adjust this to your own liking.

Automatically Wake Up

Whenever any of the computers in the house are turned on, they send a Wake-on-LAN signal to the FreeNAS server and automatically mount the shares.

Mac OS

Python is required to run this script! Basically, these scripts will check every 3 minutes that the FreeNAS server is on, and that the network shares are mounted. If not, it will attempt to turn on the FreeNAS server (by sending WakeOnLAN) and mounting afp shares. Please feel free to edit the scripts as you see fit, or change the directories they are in, of course making the necessary changes.

1) Create the file com.hoongern.nasmounter.plist in /Users/<your username>/Library/LaunchAgents

2) Create the file nas.py in /Library/Scripts/. YOU HAVE TO EDIT the configuration in the script as required:

3) Restart your Mac. If I remember, launchctl should find the new task. You can run launchctl list and check if com.hoongern.nasmounter is listed. If it’s not, I can’t actually remember what to do. I don’t actually use Mac OS, you see…

Windows

To be honest, I haven’t written a script for Windows. Not that it should be hard at all – just use a similar python script, the Task Scheduler, and the “net use” command to mount Samba shares. If there’s enough interest, I can write it up and post it here – let me know!

Linux

Again, no script yet. Shouldn’t be difficult with a similar python script, cronjob, and the “mount” command to mount Samba/NFS shares.

OpenWRT, a great alternative to DD-WRT

A short opinion on the 3rd party firmwares DD-WRT and OpenWRT, running on my TP-Link TL-WR1043ND router.

For many years now, I’ve been running DD-WRT on my Linksys WRT54GL & WRT54G2.2 routers. The two routers have lasted for over 6 years, but recently they started to develop certain issues such as wireless dropouts and slow ping times (including packet losses).

As a replacement, I decided to go the cheap route and get a TP-Link TL-WR1043ND router. To be honest, my home network requirements are pretty small. I didn’t care too much about wireless throughput as most of the network is wired gigabit. The only features I needed were:

  • Flexible port-based VLAN tagging
  • Full dnsmasq control, for assigning hostnames/domain to local machines
  • Dynamic DNS client (for dyndns.org)
  • Remote SSH management

To be honest, I have managed to achieve much of this via DD-WRT, but the whole interface is messy, with various settings being in seemingly random places. In particular, VLAN/port based tagging was so confusing via the Web UI, and CLI management (via SSH) felt disconnected and more of a “hack”, as many commands have to be saved as start-up scripts.

Enter OpenWRT, which may not have as “nice looking” a Web UI (though more than enough, in my opinion). What draws me is the neat layout of the OS. All configuration is done by editing config files directly in the file system. In addition, all the configuration options are nicely separated into different files such as ddns, network, wireless, dhcp, firewall, etc. The flexibility is nearly limitless, and because one is able to give names to interfaces and reference them in other configuration files, things are far less confusing. Configuring the switch was a breeze via the Web UI, and I was able to set up my Unifi internet connection easily to split up Internet and IPTV.

OpenWRT feels much more like a fully-fledged OS. For example, there is direct access to iptables. All I need to do is to change /etc/config/firewall, and when I’m done, just restart it with /etc/init.d/firewall restart, just like in any other Linux installation. Also, OpenWRT has built in package management, with the ability to install more packages (Yes, DD-WRT does have similar ability with optware). I quickly added WebUI Wake-On-LAN and OpenVPN.

To be honest, there are a ton of other things you can do with OpenWRT, and this messy post isn’t really a thorough comparison of the two. But having used both DD-WRT and OpenWRT, I have to say that when you need flexibility and getting the most out of your router, OpenWRT definitely trumps DD-WRT (which is still a good 3rd-party firmware, no doubt, but more useful for general/easy tasks, or for nice graphs). Once you have grasped the basics of how to manage an OpenWRT installation, you’ll be amazed how the configuration is so much easier to apply and how it works.

FreeNAS: Simultaneous AFP/CIFS shares done neatly

Learn how to hide Mac OS specific files from being displayed to a Windows client when using simultaneous AFP and Samba/CIFS shares.

In many networks these days, you’ll probably have more than just Windows or Mac OS clients. If you regularly exchange data between Mac OS and Windows/Linux using a flash drive and have hidden files enabled, you may notice a bunch of .DS_Store folders and other various files beginning with a dot. Personally, I find it somewhat annoying that Mac OS litters whole file systems with these files (In the same way, I also hate those thumbs.db files Windows generates)

In a unified file server which serves multiple operating systems, we ideally want each client to have a good experience browsing for files. We don’t want Windows users to be bogged down in a mess of files they have no clue about, and we don’t want them accidentally deleting files which may be important to the Mac OS experience.

There are no adjustments which need to be made on your AFP shares, since only Mac OS clients access those shares and know what to do with these files. However, these files should be hidden on CIFS shares. Here’s how you do it in FreeNAS 8 (The solution applies to anyone using CIFS, not just in FreeNAS):

1. Navigate to your CIFS share and press edit

2. Find “Auxilary Parameters at the bottom”

3. Enter the following text in the box

veto files = /Temporary Items/.DS_Store/.AppleDB/.TemporaryItems/.AppleDouble/.bin/.AppleDesktop/Network Trash Folder/.Spotlight/.Trashes/.fseventd/
delete veto files = yes
hide dot files = yes

Vetoing a file will render it completely invisible to anyone accessing the share. It differs from hidden files in that hidden files are hidden on the client side, but vetoed files are hidden on the server side even before the list of files is sent to the client.

The first line veto files specifies a list of file names to veto, each entry separated by a ‘/’, and you can also use wildcards (‘*’ and ‘?’) to specify multiple files with a single pattern.

“delete veto files” allows CIFS to delete any vetoed files within a directory when that said directory is deleted. If this is not set to ‘yes’, deleting a directory could fail.

“hide dot files” is optional, but simply sets all files beginning with a dot (which are hidden files in Mac OS and Linux) with a hidden flag.

4. Restart CIFS from your Services tab

If all is well, browsing your shares from all computers should now be a pleasant experience!

If you’re ever in doubt, or if you want to find out what other options can be entered, do check out the smb.conf documentation.

Network File Server using FreeNAS (Part 1)

Juggling around files can be a nightmare when you have multiple computers, especially when you use multiple operating systems. Trying to back up files can easily turn into a mess once you have multiple copies of files here and there, and you may constantly worry about hard drive failure.

By building a NAS yourself, you can solve these issues and save a lot of money compared to buying a commercial NAS product!

Juggling around files can be a nightmare when you have multiple computers, especially when you use multiple operating systems. Trying to back up files can easily turn into a mess once you have multiple copies of files here and there, and you may constantly worry about hard drive failure.

These are some of the reason which drove me to set up a NAS (Network Attached Storage) server to serve my files across my network. My requirements were the following:

  • At least 2TB of storage
  • Able to serve Windows, Mac OS, and Linux (SMB, AFP, NFS) at gigabit speeds
  • Low power consumption for 24×7 usage
  • Protected against hard drive failure
  • Protected against accidental file deletion
  • As cheap as possible

There are various companies which provide NAS solutions – for example, Synology and QNAP, as well as regular hard drive / network device manufacturers like Western Digital and Buffalo. However, as I looked through their products, I realized that while they looked very nice, they tended to be expensive and limited (depending on how deep your pockets are). For example, the Synology DS411J and QNAP TS-410 (both 4-bays) both cost around USD360, and once you’ve got them, they don’t tend to be that flexible.

So, I decided to DIY a server. Firstly, I looked at OS solutions, and settled on FreeNAS 8 (Using something like Solaris/FreeBSD would be even more flexible, but for simplicity, I went for FreeNAS which is small and has a decent web interface). The main attraction I was looking for was ZFS, a file system built to ensure data integrity. It can be thought as a marriage between hardware & software RAID, which is very beneficial because it avoids many of the write hole problems with a RAID system, and can self-heal your data. I won’t go into the details of ZFS, which can be found elsewhere, but it is clearly a very capable file system.

ZFS would provide me the following:

  • RAID-Z1: By using one of the drives in a pool for parity, I would be safe from a single hard drive failure.
  • Dataset quotas: Ability to limit datasets (or to effect, the shares) to a certain size.
  • Snapshots: You can think of this like Time Machine for Mac OS, although, to me, even better, because snapshots are instantaneous, and provide me safety against accidental file deletion, etc. They don’t take up any extra space, and allow me to roll back my file server to a particular state. Here’s how to use snapshots with Windows & Mac OS [Coming soon]
  • On-the-fly compression: Using fast compression algorithms (You can actually choose from a range – Gzip/LZJB), data can be compressed when written to the hard drive, saving space and in some cases, enabling even faster speeds on the hard drive. This is because most reading/writing is limited by hard drive speed and not CPU compression speed (At least with a modern CPU).

And having a DIY NAS server would give me the following:

  • A fast CPU for on-the-fly compression/transcoding – which I can upgrade in the future if needed
  • Flexible amount of RAM
  • Up to 6x SATA HDDs, plus with PCI-E expansion, I could easily put up to 15 Hard drives in total, for a LOT less cost than a commercial NAS
  • Ability to upgrade to 2x1Gbps network interfaces (or more), to double my network bandwidth
  • USB3.0 support

Of course, a DIY solution would tend to consume a bit more power than a NAS solution, given that it has a lot more CPU/RAM, and a lot depends on the power supply efficiency.

Parts & Build

In the end, I decided to go with the following:

  • Intel G620 2.6GHz Processor: The cheapest Socket 1155 CPU I could find at the time. It’s already a lot faster than needed, although if you do use on-the-fly encryption or compression, you can saturate it
  • Intel DH67BL-B3 microATX Motherboard: One of the cheapest boards I could find with at least 5x SATA ports. I decided to go with this because it had an Intel network controller, and according to some reviews I read, the lowest power consumption. It also gives me USB3.0 support which could be useful in the future.
  • 2x4GB Kingston DDR3-1333MHz RAM: Since RAM is cheap, and ZFS benefits from higher levels of RAM, 8GB made sense. The motherboard, supporting up to 32GB of RAM, can be easily upgraded in the future
  • 3x2TB Western Digital WD20EARX “EcoGreen” Hard drives: Not nearly the fastest drives around, but from tests (at least of the earlier generation WD20EARS), they appear to have the lowest power consumption
  • Corsair 4GB Flash Voyager USB flashdrive: One I had lying around to install FreeNAS on

All this added up to a grand total of USD422, hard drives included (USD196 without). I didn’t have a free case lying around at the time, so I just built it open:

After updating the BIOS, adjusting fan speeds, etc., I installed FreeNAS 8.0.1-RC1 onto my flash drive and booted up. Everything was detected without problems and the web interface was accessible. I created a RAIDZ1 array using the 3 drives, resulting in 3.56TB of usable space.

(Note: I had to use the wdidle3 tool to ensure my drives wouldn’t load/unload their heads so often, which could lead to premature failure)

Performance

Using the linux dd tool, I did a quick benchmark of the hard drives, which gave 217.7MB/s writing and 226.1MB/s reading. More than fast enough for me, given that gigabit LAN maxes out well before that anyway.

Next, I tested the network performance using iperf, and with 2 clients connected, the interface managed to push 950Mbps (119MB/s). All was looking well, so I quickly set up a CIFS/SMB (Windows) share and copied a file over. Both ways, I was getting 112MB/s. I also set up an AFP (Apple) share which got me the same 112MB/s.

Conclusion

I managed to get a file server which has great data integrity features, and can serve all my network clients. In addition to the regular network shares, I also use it as a “Time Capsule” of sorts for the two Mac OS machines on my network. In particular, for much less than what it would cost to buy an equivalent NAS or Time Capsule!

FreeNAS is still a work in progress, and isn’t perfect right now (still waiting for them to add VPN and torrent support), but it’s definitely on its way.

Further

FreeNAS: Simultaneous AFP/CIFS shares done neatly

Home network setup

Recently, I made the change to fiber optic for internet, boosting my download/upload speeds to 5Mbps/5Mbps. At the same time, I decided that it was time to do some upgrades to the network in my house, in particular a gigabit network and having all computers on a domain. In terms of complexity, of course this is a very small network setup compared to those you get in large companies and campuses, but for a home network, it’s probably more than an average user may have.

Recently, I made the change to fiber optic for internet, boosting my download/upload speeds to 5Mbps/5Mbps. At the same time, I decided that it was time to do some upgrades to the network in my house, in particular a gigabit network and having all computers on a domain. In terms of complexity, of course this is a very small network setup compared to those you get in large companies and campuses, but for a home network, it’s probably more than an average user may have.

Unifi, the service I’m using, provides a combination of Internet, VOIP (Phone), and IPTV in one package. This is achieved through VLAN tagging, in which various services are assigned a VLAN Tag ID so that the packets coming in can be differentiated from one another.

The fiber modem is provided with the service. This is then connected to a provided D-Link DIR615 router. Initially, the router is setup to handle all routing and connections, but since I prefer DD-WRT, I decided to use it solely as a VLAN Bridge (i.e. it removes the VLAN tags from incoming packets, and sends internet packets to Port 3, and IPTV packets to Port 4). In addition, I changed its IP/Subnet to the same subnet as the rest of my network and disabled its DHCP server, so that I can manage it from any computers on my network.

All routing is handled by my main router (a WRT54GL 1.1 running DD-WRT, overclocked to 250MHz with a heatsink mod), and is setup on a 10.0.0.0/255.0.0.0 (Class A) subnet. I made this decision after my previous Class C (255.255.255.0) network became a mess. Now, all IP addresses are nicely organized into the following pools:

10.0.0.1-10.0.0.100: Routers and bridges
10.0.0.100-10.0.0.150: DHCP clients
10.1.0.1-10.1.0.255: Static LAN clients
10.1.1.1-10.1.1.255: Static WLAN clients
10.1.2.1-10.1.2.255: Static clients with Virtual/Other Interfaces
10.2.0.1-10.2.0.255: VPN clients

Thus, for a computer which has both LAN & WLAN, the last digit [octet] of the IP address is the same. It’s LAN address would be 10.1.0.34 while its WLAN address would be 10.1.1.34. If it had a 3rd interface, it would be 10.1.2.34, etc. This makes it much neater. Of course, not wanting to memorize all the IP addresses, I decided to use DNSMasq for DNS services, with the following settings:

strict-order
domain=lan
local=/lan/
expand-hosts
address=/router.lan/10.0.0.1
address=/vpn.lan/10.0.0.2
address=/vlan615.lan/10.0.0.3

This allows me to access other machines by simply adding their hostnames to DNSMasq. This helps a lot for my machines running Web interfaces, so I can just go to http://router.lan/, etc. Provided that all clients use the router for DNS queries, it all works. Also, strict-order was required because I use Google DNS for my other DNS queries but also keep an ISP DNS server as backup – and this ensures that it uses DNS servers in the order I specified, rather than randomly (?) selecting one or the other.

A 2nd WRT54g router (running DD-WRT) adds OpenVPN support in my network, allowing me to access my network from all over the world. Ideally, my main router would run this VPN service, but as the WRT54g is an old router, it doesn’t have enough RAM (only 16MB) and processing power to achieve all this. In the future, I hope to offload this to my NAS Server.

Finally, my wired ethernet devices are mostly plugged into the last device – an 8 port gigabit switch (HP Procurve 1410-8g). This required the most work, including climbing in the attic to lay Cat-5e cable to various parts of the house (upstairs), as well as plastic channels carrying Cat-5e downstairs. Originally, I was going to get a managed switch (HP Procurve 1810-8g) so that my NAS (File) server would use an aggregated link for 2Gbps of bandwidth, but due to the extra price, I decided to just go with the unmanaged switch.

Having gigabit makes a world of difference – transferring data between various computers is much faster, a feature especially useful when I built my NAS/File server.

Some may say that my routers (WRT54g) are fairly old, and yes, that’s very true. However, in all my internet usage, I haven’t encountered any issues with them, or any internet slowdowns. I was thinking of upgrading to wireless-N, but since most of my devices are already on the wired gigabit network and most clients are only G-capable, I decided that for now, it’s not worth it.

For those wanting to setup unifi, do check out rizvanrp’s Unifi handbook.

Here’s a basic network map of my setup: