BOSCO v1.1 Features: SSH File Transfer
BOSCO v1.1 Feature: SSH File Transfer
What is it?
SSH File Transfer feature improves the method of staging input and output files to the remote cluster. In 1.0, files are transferred by starting a daemon on the remote cluster that connects back to the submit host over a random port. This required a lot of open ports on the submit host.
The new SSH File Transfer will limit the number of ports required on the submit host. BOSCO will now transfer files over a port that is forwarded over the SSH connection that BOSCO maintains with the remote cluster. The transfers are inherently secure as they are over the SSH connection, as well as they are authenticated by the Condor daemons on either end of the connection (remote cluster and submit host).
This fits into the BOSCO team's goal of lowering the amount of ports used by Condor. Our eventual goal is using the Shared Port Daemon to limit the required ports to 1 for BOSCO on the submit host.
Why should I care?
This will greatly reduce the number of ports required if you are only using the universe=grid method of submitting jobs. In fact, it will reduce the open ports on the submit host to 0. That means no more configuring firewalls for BOSCO (no campus factory support, see below). Additionally, there is no new configuration required for this feature, it 'just works' (famous last words?)
The Campus Factory, which adds features such as fault tolerant Condor file transfer and transparent multi-cluster support, still requires multiple open ports in the firewall. Additional effort will be required to change the Campus Factory configuration and daemons to support the single port. I hope that a single port will be all that is needed for v1.2.
What's Next
Over the next couple weeks, I hope to write more about upcoming features such as:
- Multi-Platform support (ie cluster and submit host are different platforms)
- Mac OSX Support
- Improved Multi-User support
Leave a comment