atppp’s Blog

文件名编码的问题能把人搞疯掉，现在我自己的网络里面，除非特别有必要，文件名的编码都一律用 UTF-8。尽管如此，各个系统之间互相操作还是一不小心就出乱码……

首先是 windows 文件共享/samba。Windows 自动用 UTF-8 没问题，smb.conf 需要配置 unix charset = UTF-8，smbmount 貌似自动用 UTF-8，Mac 下面也没问题。

然后是 shell/文件管理器下面看文件。这个一般只要 locale 设置为 UTF-8 就没有问题。如果从 windows 连上 MacOS/Linux 的 shell，还需要注意客户端的编码设置为 UTF-8 解码。MacOS shell 下用 ls 注意要加 -w 或者 -v 参数。

Secure file transfer 的话，MacOS/Linux 已经有 sftp server 了，剩下的问题都可以用 FileZilla 解决掉。Windows 架服务器用 FileZilla Server，选用 SSL/TLS support，允许并强制使用 explicit SSL/TLS，需要自己做一个证书。客户端的话，无论 Windows/MacOS/Linux 都可以用 FileZilla Client（版本 > 3.0），连 windows FileZilla Server 需要注意选择 ServerType: FTP over explicit TLS/SSL，连 MacOS/Linux 只要选用 sftp 即可。以前各个系统互通的时候 UTF-8 文件名可以搞死人，现在用 FileZilla 这套东西完全没有问题了。

补充一下，MacOS/Linux 如果在 UTF-8 locale 下，使用 lftp 命令行也很好，无论 FTP over explicit TLS/SSL 还是 sftp 都没问题，UTF-8 文件名也没问题。另外，MacOS 的 Fetch 改一下选项可以正确解码 FTP 上的 UTF-8 文件名，但是 sftp 貌似不行……还有，FileZilla Client 在 MacOS 下面貌似 bug 还比较多，希望越来越好。

没有乱码的日子很快乐。FileZilla 一统天下，UTF-8 千秋万代！

rsyncrypto is basically rsync plus encryption. It’s not feature-complete I guess, but it works for me. Ubuntu source has binary package but compiling isn’t hard at all. Alright, so obviously man page has most of what you need, here I’ll write simple outlines.

Suppose your data source folder is ~/src, sync destination is ~/dst (this could be a mounted nfs or smb share), and rsyncrypto working directory is ~/rr.

A. Encryption

First of all, create backup keys:

openssl req -nodes -newkey rsa:1536 -x509 \
  -keyout ~/rr/backup.key -out ~/rr/backup.crt

Then the actual sync command is:

rsyncrypto --ne-nesting=2 --trim=3 -n ~/rr/map \
  -cvr ~/src/ ~/dst/ ~/rr/keys ~/rr/backup.crt

This command will encrypt filename and folder structure. So obviously folder modification time will not be preserved, and empty folder will be lost. Parameter --trim depends on your folder location. --ne-nesting depends on the number of files you will have in the future. In general, you may want less than 1024 files per folder, so you need 1024*16^n > total number of files, --ne-nesting=2 is enough for 200,000 files. After this encrypted sync, and suppose you lose everything except files in the ~/dst/, then you’ll need backup.key to cold decrypt all files. So be sure to keep backup.key at another place, preferably at another physical location. You can put the above command into crontab if -v is removed. Also, syncing files without ownership might result in errors, and it may be hard to debug because rsyncrypto does not have very friendly error messages - in which case you may try to fix the file permission problem, or you may run the command by root.

If you want a parameter similar to --delete in rsync, the corresponding parameter in rsyncrypto is --delete-keys. The parameter --delete alone in rsyncrypto does not remove filename mapping in ~/rr/map.

B. Decryption

Suppose you want to cold decrypt to folder ~/rec/. First you need to decrypt filename mapping:

rsyncrypto -d ~/dst/filemap ~/rec/filemap \
  ~/rec/filemap-keys ~/rr/backup.key

The above command generates ~/rec/filemap, so then you can decrypt all files:

rsyncrypto --trim=3 -n ~/rec/filemap \
  -vrd ~/dst ~/rec/files ~/rec/keys ~/rr/backup.key

C. Windows sharing time precision

When you sync to or from windows sharing, you need a parameter similar to --modify-window=2 in rsync, but unfortunately rsyncrypto doesn’t have it. You can change source code in file.cpp, search for st_mtime, and change

dststat.st_mtime!=status.st_mtime

to

abs(dststat.st_mtime-status.st_mtime)>2

There’s another similar line you need to change in the same file.