Go to VirtualBox Preferences and under "Network", add host-only network. This will add a new vboxnet0 interface, which comes up as "192.168.56.1/24"
In the Guest Settings, add the "Host-only" to a second "Network Adapter"
Once Guest boots up, assign a static IP to the second adapter.
For Debian based servers, edit /etc/network/interfaces and add the below, which should bring up the network upon reboot:
iface eth1 inet static
You should now be able to ping and ssh to the guest via the host server on the second interface.