File transfer performance on Synology NAS

Since I've seen a bunch of questions on various NAS forums on data transfer performance, here's a quick post on the performance I observe transferring data between a Win 7 box (with source data on SATA3 external storage) to a Synology NAS DS1511+. The below are a couple of simple screen grabs taking about half an hour into transferring a full back-up using Synology DataReplicator 3. The Win7 machine and NAS are connected via gigabit ethernet to the same router. No jumbo frames or trunking.

Fig 1.  DSM 4.1 monitoring during backup

 

Fig 2. Windows 7 monitoring during backup.

 

I found that enabling jumbo frames on my Win7 machine killed any connection to the NAS, despite the NAS having jumbo frames enabled. I think (but haven't confirmed) that the router doesn't support jumbo frames, and therefore may be the cause of the issue. Having said that, it's not clear how much additional performance you could milk out of such a configuration.

 

The DSM Resource Monitor sampling frequency appears lower than the Windows monitor which may go some of the way to explaining why it looks a little more choppy as seen from the NAS. Nevertheless, sustained transfers seem to average about 80MB/s which will get through most home network transfers pretty quickly. I have noticed on some lengthier transfers that throughput can (infrequently) drop off markedly for a time before returning to about 80MB/s but haven't investigated why this occurs.

 

Backing up Ubuntu using Deja-Dup (inc. system files)

When I got the NAS set up on the network, I started playing around with Deja-Dup, the backup utility for Ubuntu. Previously I'd never backed up my Ubuntu installation but with the NAS now online it made sense to try it out. Note that Deja-Dup is a file-based back-up utility. If you want low-level backups of your partitions then you would consider using dd together with the relevant device files as described here. Be cautious with these low-level techniques as you really need to know what you're doing and can do untold damage if you don't.

Initially I ran the utility under my own account but noticed that there were a number of (system) files that didn't back up successfully due to a lack of permissions so determined that running the backup in two parts was the way to go: one as root for purely system files (NB: edit at end of post), and the other under my regular user account for anything of personal import.

Deja-dup needs to be set up for the root user. You can either do this by logging in as root if you've enable such a login in Gnome. More likely you haven't done this (Ubuntu doesn't do this for you by default), so use gksu as follows (either should work):

user@host: gksu gnome-control-center deja-dup
or
user@host: gksu deja-dup-preferences

Then configure the storage location. Mine is NFS mounted like this:

/mnt/nas/Backup/DejaDup.hostname.user

Then add/remove directories as required. To keep this as a system-specific backup I've pruned a few directories from the generic file system layout, but you can always refine this further:

Included:
/
/root


Excluded (this list includes some kernel file systems for interrogating certain kernel data structures that we don't want to back up, as well as stuff that wouldn't ever need to be restored or will be covered by non-system user backups):
/dev
/home
/mnt
/media
/proc 
/run
/sys
/tmp


A frequency of weekly and retention period of 6 months should be fine considering how often changes occur to the OS through the Ubuntu software updater and LTS releases. On my vanilla Ubuntu 12.10 desktop installation with NFS mounted backup dir running over gigabit ethernet, the backup took about 7 minutes and generated 1.2GB of backup files.

You can now run the Backup utility under your regular user account to cover off any personal files that you care about.

A final note regarding NFS mounts of your backup location. I encountered permission issues during backup writing to the backup target. You may need to investigate how the NFS server has set permissions at the target end to ensure everything works smoothly. In my case, root on the NFS client host was mapped to a different id on the NFS server. You may need to override this on the NFS server.

REDACTION: This doesn't automate backups for root or other users. Backups only seem to run for users when logged in.

I logged in to double-check the back-ups were working as expected for root (per the description given above). Whilst the configuration I had created was all still there, the backups weren't actually running automatically - presumably since you need to be logged in for the backup to get triggered. The Deja-Dup dialogue basically said something like "Last backup: 57 days ago". Anways, the back-end to Deja-Dup is duplicity, a command line utility. Depending on the selected options, Deja-Dup's auto-generated duplicity command line could look like this:

/usr/bin/python /usr/bin/duplicity --exclude=/mnt/nas/Backup/DejaDup.goat-lin.root --include=/root/.cache/deja-dup/metadata --exclude=/proc --exclude=/sys --exclude=/run --exclude=/dev --exclude=/home --exclude=/media --exclude=/tmp --exclude=/mnt --exclude=/sys --exclude=/proc --exclude=/tmp --exclude=/root/.gvfs --exclude=/root/.cache/deja-dup --exclude=/root/.cache --include=/ --include=/root --exclude=** --gio --volsize=50 / file:///mnt/nas/Backup/DejaDup.goat-lin.root --verbosity=4 --gpg-options=--no-use-agent --archive-dir=/root/.cache/deja-dup

Now you can go put something similar to the above in cron. Edit 7/7 to remove some uneccessary options and reduce verbosity.

DHCP on Synology NAS (dhcpd and dnsmasq)

As an adjunct to my home network DNS configuration, I decided to move DHCP duties across to my always-on Synology NAS (DSM4.1) and disable the DHCP services on my routers. This was driven by a number of (not particularly compelling) factors, however my experience follows. This is as much a story about making mistakes as getting it right, so read through to the end before you start cutting and pasting commands into your live environment.

As Synology offers a DHCP server package, I installed this on the base installation via the DSM Package Centre utility. A new icon will appear in your apps list and the Control Panel->Networks applet gets a new 'DHCP Server' button. A form asks you to configure key details for your DHCP server, including primary and secondary DNS, domain name, lease time and so forth. You can also configure subnets, static IP mappings and other details. Neither of my routers support a particularly sophisticated DHCP management interface so this is one somewhat compelling reason to move away from allowing a vanilla consumer router to provide DHCP services on your network.

So the moment of truth arrives. Knowing that my DNS configuration on the NAS works I switch on DNS on the NAS and disable it on the routers. Needless to say, there were issues. The DHCP server on the NAS was broken but it wasn't obvious why, and now my devices were having issues getting onto the network (including the admin interfaces to my router and NAS...).

Running 'ps | grep dhcpd' shows that dhcpd is not running. This is despite no warnings or errors being issued by the Synology UI. The first stop is /var/log/messages for any clues that might have been left.

Nov  3 21:42:18 dhcpserver.cgi: net_get_dhcp_server_conf.c:164 File [/etc/dhcpd/dhcpd-bond0-bond00.conf] does not exist.
Nov  3 21:42:18 dhcpserver.cgi: dhcpserver.cpp:130 Cannot read Global setting on bond0 bond00
Nov  3 21:42:19 dhcpserver.cgi: dhcpserver.cpp:317 Can not open DHCP static file
Nov  3 21:42:19 dhcpserver.cgi: dhcpserver.cpp:410 Can not open DHCP lease file
All the files that are claimed not to exist are clearly extant:
dns> ls /etc/dhcpd
dhcpd-bond0-bond00.conf  dhcpd-static.conf        dhcpd.conf               dhcpd.info

I then tried to hunt down the init.d management scripts and dhcpd binary but these were nowhere to be found on the system. So I have basically let myself up the garden path as I'm not looking for the right thing. You can install regular old dhcpd if you like via ipkg install dhcp, but it won't work with the Synology's UI (web management interface) - at least not without some hackery. It didn't initially twig for me that under the hood Synology's DHCP Server uses dnsmasq ("a lightweight, easy to configure DNS forwarder and DHCP server"). Admittedly, whilst I had seen references to dnsmasq I did not know what it was precisely and it wasn't until I dug around in places like /etc/rc.network and recalled some error entries in /var/log/messages like the below that it made sense that dnsmasq was in use and why it wouldn't run.

dnsmasq[22230]: failed to create listening socket for port 53: Address already in use
dnsmasq[22230]: FAILED to start up

If you had previously installed the dns package, it would listen on port 53 and would prevent dnsmasq from starting. You may also have noticed in Synology's UI that although it reports that DHCP Service is running in Package Center, every time you navigate back to Control Panel->Network->Network Interface->DHCP Server, under the General tab the 'Enable DHCP Server' check box is always unchecked even if you see the 'Settings applied' message after clicking Apply, implying to me at least that it wasn't actually running.

If you're running named, kill it:
/opt/etc/init.d/S09named stop

Under /var/log/messages you may also see something like:
dhcpserver.cgi: dhcpserver.cpp:410 Can not open DHCP lease file

There is a post on the Synology site that has a simple remedy for this:
touch /var/packages/DHCPServer/target/etc/dhcpd.conf.leases
touch /var/packages/DHCPServer/target/etc/dhcpd-leases.log

On my NAS, only dhcpd-leases.log wasn't present so I ran the second command only and dnsmasq came up cleanly using the Synology UI. Check that it's running:

dns> ps | grep dnsmasq
20315 root      1620 S    dnsmasq --user=root --cache-size=200 --conf-file=/etc/dhcpd/dhcpd.conf --dhcp-lease-max=2147483648
20316 root      1620 S    dnsmasq --user=root --cache-size=200 --conf-file=/etc/dhcpd/dhcpd.conf --dhcp-lease-max=2147483648
A few more tips here:

• within the Synology UI you need to ensure that under the table headed 'Subnet list' that the row(s) you have entered have green check marks next to them. It doesn't seem to check this box by default and will not work if unchecked.
• (obviously) you need to turn off other DHCP server(s) on your network as appropriate. Chances are your router is running DHCP which will interfere with things.
• consider how to manage addresses on your network. There are going to be a number of hosts that you will probably want to have static IP addresses (NAS, routers, other servers), but the rest can be dynamic. Use start/end addresses and reserved addresses to ensure you have enough addresses of each type and don't overlap between the static and dynamic address pools.

Testing out the operation of the dnsmasq DNS server reveals that it pretty much works like the old named server however there are a few minor updates to /etc/hosts to capture the static addresses and hosts on your network. This is easier than mucking about with forward and reverse lookup files in BIND IMHO.

• after config changes, restart dnsmasq (or the DHCP Server under Package Center)
• the Synology implementation looks to regenerate /etc/dhcpd/dhcpd.conf when you make changes in the DSM UI. dnsmasq supports a lot of options, so you may need to look into how to preserve any extended config you intend to remain persistent.

[Edit 25/11/2012]: I upgraded to DSM 4.1-2661 which caused some silly issues.

• I didn't disable my named package. When the NAS upgrade completed and the system rebooted, my old named config was left lying around which meant /opt/etc/init.d/S09named was called and prevented dnsmasq from starting (both need port 53). All dns and dhcp services were down as a result and therefore no internet access until this got fixed, made all the more annoying as none of my computers could get on the network without manual intervention.
• /etc/hosts looks like it got touched during the upgrade. I have no proof, but some static host/IPs configured in this file appeared to have disappeared. When I readded them and restarted dnsmasq these hosts would resolve properly on the network again.

DNS configuration for your home network

Installing BIND on the NAS

 

I found some documentation on how to set up BIND (DNS) on a Synology NAS running Linux but as I ran into some problems I thought I'd document them here on the offchance someone finds it a useful reference. [Edit]: Other (simpler and in many ways better) ways of doing this exist.

 

My starting point was here, but quickly found that the more comprehensive documentation here was also useful. BIND configuration notwithstanding, the installation of the BIND package on the Synology NAS (DSM 4.1) was not without issues.

 

You start by installing the BIND package:

DiskStation> ipkg install bind
Installing bind (9.6.1.3-4) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/bind_9.6.1.3-4_i686.ipk
Installing openssl (0.9.8v-2) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/openssl_0.9.8v-2_i686.ipk
Installing psmisc (22.17-1) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/psmisc_22.17-1_i686.ipk
Installing ncurses (5.7-1) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/ncurses_5.7-1_i686.ipk
Configuring bind
Running post-install
You must now create your named.conf file
When it is installed in /opt/etc/named/named.conf, execute
/opt/etc/init.d/S09named start
to start service
You will probably also want to create rndc.conf by running
rndc-confgen. Of course, you may want to set your name server
in the GUI to 127.0.0.1 or your local ip-address
Configuring ncurses
update-alternatives: Linking //opt/bin/clear to /opt/bin/ncurses-clear
Configuring openssl
Configuring psmisc
update-alternatives: Linking //opt/bin/killall to /opt/bin/psmisc-killall
update-alternatives: Linking //opt/bin/pidof to /opt/bin/psmisc-killall
Successfully terminated.

I created /opt/etc/named/named.conf and related files per my desired set-up and tried to start the daemon:

DiskStation> /opt/etc/init.d/S09named start
Starting DNS Services: /opt/bin/pidof: error while loading shared libraries: libssp.so.0: cannot open shared object file: No such file or directory
started
The forums will tell you that you need gcc installed to have access to this library, so go do it. Before you do it, however, make sure root's PATH environment variable has /opt/bin and /opt/sbin at the START (that is, edit and source ~/.profile):

PATH=/opt/bin:/opt/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/syno/sbin:/usr/syno/bin:/usr/local/sbin:/usr/local/bin

Otherwise you will encounter errors like the ones below:

DiskStation> /opt/bin/ipkg install gcc
Installing gcc (4.2.1-5) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/gcc_4.2.1-5_i686.ipk
file_move: ERROR: failed to rename /opt/ipkg-Ys4oOA/gcc_4.2.1-5_i686.ipk to /opt/ipkg-Ys4oOA/gcc_4.2.1-5_i686.ipk: No such file or directory
Nothing to be done
An error ocurred, return value: -1.
Collected errors:
Failed to download gcc. Perhaps you need to run 'ipkg update'?
It appears that there are at least two wget binaries installed on the system and the Synology version doesn't work with ipkg.
/usr/syno/bin/wget (GNU Wget 1.10.1)
/opt/bin/wget (GNU Wget 1.12)

DiskStation> ipkg install gcc
Installing gcc (4.2.1-5) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/gcc_4.2.1-5_i686.ipk
Installing binutils (2.19.1-1) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/binutils_2.19.1-1_i686.ipk
Installing libc-dev (2.3.6-5) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/libc-dev_2.3.6-5_i686.ipk
Installing libnsl (2.3.6-4) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/libnsl_2.3.6-4_i686.ipk
Configuring binutils
update-alternatives: Linking //opt/bin/strings to /opt/bin/binutils-strings
Configuring gcc
Configuring libc-dev
Configuring libnsl
Successfully terminated.
Now let's start the daemon again:
DiskStation> /opt/etc/init.d/S09named start
Starting DNS Services: started
Well this is a lie.
DiskStation> ps | grep named
12730 root      2540 S    grep named
Check /var/log/messages to see what happened:
[...]
Nov  1 15:55:25 named[8642]: dns_master_load: root.servers:40: unexpected end of file
Nov  1 15:55:25 named[8642]: dns_master_load: root.servers:40: unexpected end of input
Nov  1 15:55:25 named[8642]: could not configure root hints from 'root.servers': unexpected end of input
Nov  1 15:55:25 named[8642]: loading configuration: unexpected end of input
Nov  1 15:55:25 named[8642]: exiting (due to fatal error)
Ok, my fault for incorrectly editing the config files (a couple of issues look related to a possible  bug using 'open' (o) command in vi which incorrectly splits the last character of the line onto a new line) so clean this up and see what else was missed by looking at /var/log/messages again:
[...]
Nov  1 16:21:02 named[12572]: none:0: open: /opt/etc/named/rndc.key: file not found
Nov  1 16:21:02 named[12572]: /opt/etc/named/named.conf:19: couldn't install keys for command channel 127.0.0.1#953: file not found
Nov  1 16:21:02 named[12572]: /opt/etc/named/named.conf:19: couldn't add command channel 127.0.0.1#953: file not found
Nov  1 16:21:02 named[12572]: logging channel 'dns_log' file '/opt/var/log/dns.log': file not found
Nov  1 16:21:02 named[12572]: isc_log_open '/opt/var/log/dns.log' failed: file not found
Silly mistakes corrected, but it is at least now running with some other issues:
DiskStation> ps | grep named
12572 root      7324 S    /opt/sbin/named -c /opt/etc/named/named.conf
12730 root      2540 S    grep named
Thank goodness for that. Now let's check if the config works.

dns> nslookup
> server 192.168.1.2
Default server: 192.168.1.2
Address: 192.168.1.2#53
> dns.mydomain.net
[...]
Name:   dns.mydomain.net
Address: 192.168.1.2
> bogus.mydomain.net
[...]
** server can't find bogus.mydomain.net: NXDOMAIN
> router.mydomain.net
[...]
Name:   router.mydomain.net
Address: 192.168.1.1
> r6300.mydomain.net
[...]
r6300.mydomain.net        canonical name = router.mydomain.net.
Name:   router.mydomain.net
Address: 192.168.1.1
> www.google.com
[...]
Non-authoritative answer:
Name:   www.google.com
Address: 74.125.237.82
Name:   www.google.com
Address: 74.125.237.83
Name:   www.google.com
Address: 74.125.237.84
Name:   www.google.com
Address: 74.125.237.80
Name:   www.google.com
Address: 74.125.237.81
> 192.168.1.2
[...]
2.1.168.192.in-addr.arpa        name = dns.mydomain.net.
Cool. All the locally configured hosts are there, non-existent local hosts are not found and hosts on the internet are also found. A reverse lookup also appears to have worked. Same results on the NAS (127.0.0.1) and a Win-7 host (connecting to 192.168.0.2).