1. Using Python Virtual Environment to install new softwares without root right.
Please try to use Python
virtualenv to setup and manage Python environments
For example to use Python 3.6.3 with PyTorch 0.2.0_3, I’m working in the folder /home/wang/pyenv
(1) To create a new folder
(2)
module load pytorch/python3.6/0.2.0_3
(3)
virtualenv --system-site-packages py3.6.3
(4)
source py3.6.3/bin/activate
(5)
pip install opencv-python Pillow pywavelets scikit-learn
Now every time after run
module load pytorch/python3.6/0.2.0_3
source /home/wang/pyenv/py3.6.3/bin/activate
you’ll use this environment.
[wang@c17-04 ~]$ module load pytorch/python3.6/0.2.0_3
[wang@c17-04 ~]$ source /home/wang/pyenv/py3.6.3/bin/activate
(py3.6.3) [wang@c17-04 ~]$ python -c "import torch; print(torch.__version__); print(torch.__file__)"
0.2.0_3
/share/apps/pytorch/0.2.0_3/python3.6/lib/python3.6/site-packages/torch/__init__.py
(py3.6.3) [wang@c17-04 ~]$ python -c "import cv2; print(cv2.__version__); print(cv2.__file__)"
3.3.0
(py3.6.3) [wang@c17-04 ~]$
2. Use X graph (windows) to display
Inside NYU campus, please directly login to prince.hpc.nyu.edu without through HPC gateways with command
ssh -Y prince.hpc.nyu.edu
When you are working off campus, please try to use HPC gateways is gw.hpc.nyu.edu, /home on new gateway is mounted as read only, users can not write any data there. HPC gateways are mainly for setup ssh tunneling, after sshtunneling session is running, users can directly login to prince cluster from their local computers.
For Mac, please follow the instructions on HPC wiki to setup ssh tunneling
(1) Open an iTerm, run command
to setup ssh tunneling through gateway.
(2) Keep iTerm window in (1) open, open a new iTerm run command
you should be able to directly login to prince from your local computer as long as ssh tunneling in step (1) is running, X11 forwarding is enabled.
scp is similar as ssh.
For Windows with Putty for ssh login
For Windows with SCP for data transfer
Wireshark 2.6.3. is available with Singularity image file
/beegfs/work/public/singularity/wireshark-2.6.3.simg
Please launch wireshark with command
singularity exec /beegfs/work/public/singularity/wireshark-2.6.3.simg wireshark
[wang@c17-04 ~]$ singularity exec /beegfs/work/public/singularity/wireshark-2.6.3.simg which wireshark
/usr/bin/wireshark
[wang@c17-04 ~]$ singularity exec /beegfs/work/public/singularity/wireshark-2.6.3.simg wireshark --version
Wireshark 2.6.3 (Git v2.6.3 packaged as 2.6.3-1~18.04.0)
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Compiled (64-bit) with Qt 5.9.5, with libpcap, with POSIX capabilities (Linux),
with libnl 3, with GLib 2.56.2, with zlib 1.2.11, with SMI 0.4.8, with c-ares
1.14.0, with Lua 5.2.4, with GnuTLS 3.5.18, with Gcrypt 1.8.1, with MIT
Kerberos, with MaxMind DB resolver, with nghttp2 1.30.0, with LZ4, with Snappy,
with libxml2 2.9.4, with QtMultimedia, with SBC, with SpanDSP, without bcg729.
Running on Linux 3.10.0-514.10.2.el7.x86_64, with Intel(R) Xeon(R) CPU E5-2690
v4 @ 2.60GHz (with SSE4.2), with 128826 MB of physical memory, with locale C,
with libpcap version 1.8.1, with GnuTLS 3.5.18, with Gcrypt 1.8.1, with zlib
1.2.11, binary plugins supported (0 loaded).
Built using gcc 7.3.0.
[wang@c17-04 ~]$
singularity exec /beegfs/work/public/singularity/wireshark-2.6.3.simg wireshark
Solution:
https://uisapp2.iu.edu/confluence-prd/pages/viewpage.action?pageId=280461906
Mac OS X
- Install XQuartz on your Mac, which is the official X server software for Mac
- Run Applications > Utilities > XQuartz.app
-
Right click on the XQuartz icon in the dock and select Applications > Terminal. This should bring up a new xterm terminal windows.
- In this xterm windows, ssh into the linux system of your choice using the -X argument (secure X11 forwarding). For example, to log into hulk.soic.indiana.edu you would run something like:
ssh -Y [email protected]
- Once you are logged into the linux system, you can just run the GUI program of your choice (ie. matlab, mathematics, etc) and it will display on your Mac.
https://wikis.nyu.edu/display/ADRC/Enable+X11+Forwarding+on+Mac+OS+X
If you own a recent Mac computer and you realize that you can't start any GUI app from the cluster, even when you used the -X option with ssh; then you may have to enable X11 Forwarding on MacOS.
Step-by-step guide
-
By default, X11 forwarding is not enabled on Mac Leopard, and from Mountain Lion on, Apple decided not to ship X11 with the OS. If that latter is your case, then first you will have to install XQuartz from here: http://xquartz.macosforge.org.
-
Once you have it installed, you should be able to connect to prince with X11.
ssh -X <NetID>
@prince.hpc
.nyu.edu