SFTPGO can run SFTP, FTP and webdav and you can configure users via a web browser. In this post I set up a SFTPGO SFTP server on a A Hetzner auction server with 14 TB of storage running ubuntu 20.10.
Activate the rescue system and install ubuntu 20.10
You will now have a root password. When you reset the server with Reset – Send CTRL-ALT-DEL to the server you can log in and run installimage
Login to the server with the command ssh root@1.1.1.1 in terminal on a mac or ubuntu on windows
Run
installimage
Choose to install ubuntu 20.10. Since it is not a LTS release you can easily add the SFTPGO PPA repository later.
The default settings for this 6 TB 4 HDD server is RAID level 6 where two hard drives are redudant. 2 TB for / and almost all of the the rest for storage for /home is the default. I changed to RAID level 5 so only one hard drive is redudant and set 1080G for / and 1080G for /home and the rest in /srv (14TB) for sftpgo to use.
Install Ubuntu by exiting the editor with F10 and finish the installer
Reboot the server
Check storage with df -h
df -h
Update Ubuntu
sudo apt update
sudo apt upgrade
Add a sudoer user (optional)
sudo adduser newuser
usermod -aG sudo newuser
Login in with the new sudo user.
Disable root ssh login and change ssh port (optional)
For security it common to change the default ssh port and disable root ssh login.
Change the firewall on the hetzner server admin web site to use the alternative SSH port, SFTPGO SFTP port and web interface port
I edited the webserver template to also accept ssh/sftp on port to 2222, 2022 and http on 8080. You can remove access to the web interface in the firewall when it is not in use.
Install SFTPGO
The easiest way to install SFTPGO is to add the sftpgo ppa repository to ubuntu 20.10. In 20.04 LTS it is not allowed by default to add repositories.
sudo apt install software-properties-common
sudo add-apt-repository ppa:sftpgo/sftpgo
sudo apt install sftpgo
sudo systemctl status sftpgo
Open the SFTPGO web interface to the internet
Change “httpd” “address”: “”127.0.0.1″ to httpd” “address”: “” settings in the sftpgo.json configuration file so you can access the web ui from the internet. You can also enable defender.
sudo nano /etc/sftpgo/sftpgo.json
Restart sftpgo
sudo systemctl restart sftpgo
Change the default admin password
Login to the web admin interface with a web browser with the username admin and the password password. Use the IP address of the server and port 8080 http://1.1.1.1:8080/ Change the default admin password.
Add a SFTPGO SFTP user
If you only add username, password and * permissions the user will get a default home dir in a chrooted directory in /srv/sftpgo/data/user1. This user is not an actual user on the system and needs a home directory that the sftpgo user has access to like /srv . You can also choose to enable max connections, disk storage quota and max bandwith and more if you choose to.
You can now connect to the SFTP server with Cyberduck or similar SFTP clients. I recommend using Cyberduck because it uses segmented downloads.
Connect to the SFTPGO SFTP server
Choose SFTP and port 2022 and username and password.
Try uploading and downloading. I get 22 MB/ sec downloading on a 200 mbit connection.
You can which users are connecting and downloading when choosing connections in SFTPGO web admin.
Check CPU and memory usage on the server with htop
sudo apt install htop
htop
SFTPGO uses a litte more CPU than the Openssh SFTP server. The server seems to handle it well.
A DKDM is a KDM made for Easydcp creator + and similar programs that can do versioning like adding subtitles change audio and add logo reels to DCPs .
A KDM is for a cinema server/player that can only play the DCP.
When a distributor buys a DCP for versioning it is often encrypted and you can buy a DKDM that lets you do versioning on it. When the new version is ready you can encrypt it again.
It is possible to get the public encryption keys from cinemas that will screen a movie and make KDMs. I mostly send encrypted DCP to cinemas in Norway via Unique Movie transit and let them handle the logistics of generating and sending KDMs to cinemas. It is no extra cost to let them handle the KDMs.
This is how I make an encrypted DCP and a DKDM for a DCP lab or Movie Transit.
Convertering a DCP to an encrypted DCP in Easydcp Creator +
To enable encyption I click on the key on the video track and choose Enable for all tracks.
Now the lock icon is locked
To make the encrypted DCP I click on Generate DCP
Choose to make DCP digest in the Generate Package Wizard
To make a test KDM you can first make a KDM for your copy of Easydcp Creator.
Open the encrypted DCP in Easydcp creator. The keys are red and the DCP is locked.
Export the Public encryption key File – Content decryption – Export public certificate
Making a KDM/DKDM in Easydcp Generator
In KDM Generator choose the DCP-Digest that was created when the DCP was created. Choose the public certificate that you exported from Easydcp Creator and choose valid dates and then click on Generate KDM
In Easydcp Creator choose Export the Public encryption key File – Content decryption – Load Content Keys and load the KDM you generated.
Now the DCP has green lock icons and the DCP is unlocked.
When generating the DKDM for Movie Transit I use the public certificate they sent me and generate a key for them in KDM generator.
Here is some tips when setting up SFTP on a dedicated server running Ubuntu 20.04 or 20.10. A hetzner auction server starts at 30 euro a month. A Raspberry Pi 4/400 cost around 100 euro and can be used as SFTP server.
Start the OpenSSH server
The OpenSSH server is already running on Ubuntu server, but if you are using Ubuntu Desktop you can install and start the Openssh server
#!/usr/sbin/nft -f
flush ruleset
table inet firewall {
chain inbound {
# By default, drop all traffic unless it meets a filter
# criteria specified by the rules that follow below.
type filter hook input priority 0; policy drop;
# Allow traffic from established and related packets.
ct state established,related accept
# Drop invalid packets.
ct state invalid drop
# Allow loopback traffic.
iifname lo accept
# Allow all ICMP and IGMP traffic, but enforce a rate limit
# to help prevent some types of flood attacks.
ip protocol icmp limit rate 4/second accept
ip6 nexthdr ipv6-icmp limit rate 4/second accept
ip protocol igmp limit rate 4/second accept
# Allow SSH on port 22.
tcp dport 22 accept
# Allow HTTP(S).
# -- From anywhere
#tcp dport { http, https } accept
#udp dport { http, https } accept
# -- From approved IP ranges only
# tcp dport { http, https } ip saddr $SAFE_TRAFFIC_IPS accept
# udp dport { http, https } ip saddr $SAFE_TRAFFIC_IPS accept
# Uncomment to allow incoming traffic on other ports.
# -- Allow Jekyll dev traffic on port 4000.
# tcp dport 4000 accept
# Uncomment to enable logging of denied inbound traffic
# log prefix "[nftables] Inbound Denied: " flags all counter drop
}
chain forward {
# Drop everything (assumes this device is not a router)
type filter hook forward priority 0; policy drop;
# Uncomment to enable logging of denied forwards
# log prefix "[nftables] Forward Denied: " flags all counter drop
}
chain outbound {
# Allow all outbound traffic
type filter hook output priority 0; policy accept;
}
}
List the NFT ruleset
sudo nft list ruleset
Stop brute force attacks with sshguard and nftables
Soon you will see blocked IP addresses when you list the NFT ruleset
sudo nft list ruleset
Make chroot jail sftp users that can download and upload only from a folder in their home directory
To enable SFTP with chrooted users you need to manually edit sshd_config with the correct settings that disables shell login and then manually add chrooted users. Link: How to use SFTP with a chroot jail
Change these settings in /etc/ssh/sshd_config
nano /etc/ssh/sshd_config
Put a comment # before this line
#Subsystem sftp /usr/lib/openssh/sftp-server
Add this line
Subsystem sftp internal-sftp
add the group sftpusers
Match Group sftpusers
ChrootDirectory %h
ForceCommand internal-sftp
AllowTcpForwarding no
X11Forwarding no
Make a download user called user1 that can download and upload to only /home/user1/data
Make a user with sudo privileges instead of using root
When Ubuntu is installed you have root access and can log on to the server in a SSH shell with the root account. For better security make a user with sudo privileges that is used instead of root.
Logging on to the server with SSH and downloading a DCP folder or videofiles from another SFTP/FTP server
Sometimes it can be nice to download something directly from ftp servers and filemail via ftp to the server. In terminal you can use SSH to login to the server and use the command screen to detach from the session so you can resume it if the connection to the server is broken.
ssh user@111.111.111.111
If I want to download something to a folder /home/user/download/newfolderwiththings.
cd /home/user/download
mkdir newfolderwiththings
sudo apt update
sudo apt install ncftp
sudo apt install screen
screen
ncftp -u username 111.111.111.111
When you have started ncftp you can use get with the recursive command -R to download a folder and subdirectories. And use ls to list the content and cd to change directories,
ls
cd folder
get -R /folderwithhings
detach from the screen by using control + a,d
when connecting to the ssh server again use screen -r to attach to the session again.
screen -r
If you have started many screen sessions you will get a list of sessions so you can choose which to start. To exit a screen session you can use
exit
Compress and uncompress files on the server
You can make a 7z compressed archive of files on the server so it would be faster to download them. You can also upload an archive and then uncompress it on the server. You can also split the archive in parts so you can download many parts at once. You can use screen to be able to log of the ssh session while the files are being compressed.
Compress a directory and sub directories. Normal compression. Useful for wav files and similar files that will compress to half the size. Split in 3 GB parts. Cyberduck default setting is to use segmented download, but it can speed up uploads and downloads to split files in parts.
7z a -r -v3000m directory.7z /directory
Compress file. Using -mx0 for no compression. Split in 3 GB parts . Useful if you want to make the archive faster and for files that don’t compress much like prores video files.
7z a -mx0 -v3000m archive.7z prores.mov
You can also upload 7z archive files and extract them on the server. Extract archive recursively and keep the subdirectories
7z x archive.7z
Making a MD5 checksum file of files in a directory.
.You can check the MD5 checksum of a file on the server with the command
md5sum file
To be able to test the integrity of folder with files you can have md5 text checksum files. You can make a checksum.md5 file with the md5 checksum of files in a directory like this
md5sum * > checksums.md5
To check the checksum of the files in a directory with the md5 checksum file you can use .
md5sum -c checksums.md5
Mount a remote directory with sshfs and fuse.
Sometimes it can be handy to change the name of a DCP that had already been uploaded. Or do other things with files on the server as if they were on your local computer . With sshfs and fuse you can mount a folder on the server as a local folder and open it in Easydcp Creator and change the name or other metadata like the content kind or the offset on reels. Easydcp Creator saves the new metadata to the folder on the server. To mount the folder from the server locally on my mac I use the commands from here
To open the DCP in the Easydcp Creator I drag the folder to the Easydcp window and wait a little bit. You can also open a Resolve or Premiere project this way.
To unmount you can use the umount command
sudo umount /Users/yourname/Desktop/server
Converting a video file or sound file on the server with ffmpeg
Converting a high-quality video file to a low-res version on the server so it can be downloaded easier. Example: You need a file to check subtitles. Or a small file to upload as a screener. If you need a small size h264 video file of a file on the server you can use ffmpeg to convert it.
