Ubuntu设置HTTP与SOCKS5代理进行流量转发

需求:由于内网上网账号有登录设备数量限制,无法集群内所有物理机都登录账号,因此设置一台机器作为流量出口,设置HTTP与SOCKS5代理。

代理服务端设置

使用tinyproxy进行HTTP代理,使用danted进行SOCKS5代理。

apt install tinyproxy
apt install danted-server

此处我们设置HTTP代理端口1081,SOCKS5代理端口1080,忽略账号验证(可能不安全,建议设置)。

如果要设置账号验证,任意设置一个账号用于连接。

sudo useradd ss5 -g proxy
sudo passwd ss5
# disable login
sudo vi /etc/passwd

/etc/tinyproxy(其中许多保持默认,只需要修改port与allow即可)

##
## tinyproxy.conf -- tinyproxy daemon configuration file
##
## This example tinyproxy.conf file contains example settings
## with explanations in comments. For decriptions of all
## parameters, see the tinproxy.conf(5) manual page.
##

#
# User/Group: This allows you to set the user and group that will be
# used for tinyproxy after the initial binding to the port has been done
# as the root user. Either the user or group name or the UID or GID
# number may be used.
#
User ss5
Group proxy

#
# Port: Specify the port which tinyproxy will listen on.  Please note
# that should you choose to run on a port lower than 1024 you will need
# to start tinyproxy using root.
#
Port 1081

#
# Timeout: The maximum number of seconds of inactivity a connection is
# allowed to have before it is closed by tinyproxy.
#
Timeout 600

#
# DefaultErrorFile: The HTML file that gets sent if there is no
# HTML file defined with an ErrorFile keyword for the HTTP error
# that has occured.
#
DefaultErrorFile "/usr/share/tinyproxy/default.html"

#
# StatFile: The HTML file that gets sent when a request is made
# for the stathost.  If this file doesn't exist a basic page is
# hardcoded in tinyproxy.
#
StatFile "/usr/share/tinyproxy/stats.html"

#
# LogFile: Allows you to specify the location where information should
# be logged to.  If you would prefer to log to syslog, then disable this
# and enable the Syslog directive.  These directives are mutually
# exclusive. If neither Syslog nor LogFile are specified, output goes
# to stdout.
#
LogFile "/var/log/tinyproxy/tinyproxy.log"

#
# Syslog: Tell tinyproxy to use syslog instead of a logfile.  This
# option must not be enabled if the Logfile directive is being used.
# These two directives are mutually exclusive.
#
#Syslog On

#
# LogLevel: Warning
#
# Set the logging level. Allowed settings are:
#	Critical	(least verbose)
#	Error
#	Warning
#	Notice
#	Connect		(to log connections without Info's noise)
#	Info		(most verbose)
#
# The LogLevel logs from the set level and above. For example, if the
# LogLevel was set to Warning, then all log messages from Warning to
# Critical would be output, but Notice and below would be suppressed.
#
LogLevel Info

#
# PidFile: Write the PID of the main tinyproxy thread to this file so it
# can be used for signalling purposes.
# If not specified, no pidfile will be written.
#
PidFile "/run/tinyproxy/tinyproxy.pid"

#
# MaxClients: This is the absolute highest number of threads which will
# be created. In other words, only MaxClients number of clients can be
# connected at the same time.
#
MaxClients 100

#
# MinSpareServers/MaxSpareServers: These settings set the upper and
# lower limit for the number of spare servers which should be available.
#
# If the number of spare servers falls below MinSpareServers then new
# server processes will be spawned.  If the number of servers exceeds
# MaxSpareServers then the extras will be killed off.
#
MinSpareServers 5
MaxSpareServers 20

#
# StartServers: The number of servers to start initially.
#
StartServers 10

#
# MaxRequestsPerChild: The number of connections a thread will handle
# before it is killed. In practise this should be set to 0, which
# disables thread reaping. If you do notice problems with memory
# leakage, then set this to something like 10000.
#
MaxRequestsPerChild 0

#
# Allow: Customization of authorization controls. If there are any
# access control keywords then the default action is to DENY. Otherwise,
# the default action is ALLOW.
#
# The order of the controls are important. All incoming connections are
# tested against the controls based on order.
#
Allow 127.0.0.1
#Allow 192.168.0.0/16
#Allow 172.16.0.0/12
Allow 10.0.0.0/8

ConnectPort 443
ConnectPort 563

/etc/danted.conf

logoutput: syslog stdout /var/log/sockd/sockd.log

internal: eno1 port = 1080

external: ppp0

socksmethod: none

clientmethod: none

user.privileged: root

user.notprivileged: ss5

client pass {

from: 0.0.0.0/0 port 1-65535 to: 0.0.0.0/0

}

socks pass {       

from: 0.0.0.0/0 to: 0.0.0.0/0       

command: bind connect udpassociate       

log: error # connect disconnect iooperation

}

socks pass {       

from: 0.0.0.0/0 to: 0.0.0.0/0       

command: bindreply udpreply       

log: error # connect disconnect iooperation

}

运行

cd /etc/tinyproxy
service tinyproxy start
/etc/init.d/danted start

查看端口是否正常监听即可。如果danted遇到问题,可查看danted -V检查配置文件。

sudo netstat -anp | grep 1080

代理客户端设置

代理设置加入环境变量:

export all_proxy="socks5://IP:1080"
export ALL_PROXY="socks5://IP:1080"
export http_proxy="socks5://IP:1080"
export https_proxy="socks5://IP:1080"

为wget设置代理:~/.wgetrc

#You can set the default proxies for Wget to use for http, https, and ftp.
# They will override the value in the environment.
http_proxy=http://IP:1081
https_proxy=http://IP:1081
ftp_proxy=http://IP:1081

# If you do not want to use proxy at all, set this to off.
use_proxy = on

Build a Docker Image

为了避免遗忘,整理一下开发使用Docker配置PyTorch/TensorFlow环境的方法


DockerFile

# For more information, please refer to https://aka.ms/vscode-docker-python
FROM cddlyf/pytorch1.3.0-tensorflow-py36-cuda10.1:latest

# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE=1

# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED=1

# Copy pip requirements
COPY requirements.txt .

# Copy pre-downloaded wheels
COPY ./wheels ./wheels

# switch to ustc pip source
# RUN pip install --default-timeout=10 -U pip
# RUN pip config set global.index-url http://pypi.mirrors.ustc.edu.cn/simple
# RUN pip config set install.trusted-host pypi.mirrors.ustc.edu.cn
# RUN pip install -r requirements.txt

# Run pip install from file
RUN pip install --no-index --find-links=./wheels -r requirements.txt

RUN pip install torch-cluster==1.4.5
RUN pip install torch-geometric==1.3.2
RUN pip install torch-scatter==1.3.2
RUN pip install torch-sparse==0.4.3

# Install libraries
COPY ./mesh /mesh
# RUN apt-get install libboost-dev
WORKDIR /mesh
RUN make all

# Copy code from . to $CONTAINER/app
WORKDIR /app
COPY . /app

# Remove wheels
RUN rm -rf /app/wheels

# Creates a non-root user with an explicit UID and adds permission to access the /app folder
# For more info, please refer to https://aka.ms/vscode-docker-python-configure-containers
# RUN adduser -u 5678 --disabled-password --gecos "" appuser && chown -R appuser /app
# USER appuser

# During debugging, this entry point will be overridden. For more information, please refer to https://aka.ms/vscode-docker-python-debug
# CMD ["python", "main.py"]

一些记录:

  1. 由于pip安装时间长,经常崩溃,可以先使用pip download -d wheels -r requirements.txt将依赖wheels下载下来,再进行本地安装
  2. 配置错误产生中间文件太多,可以用docker image prune , docker container prune 清理

构建与运行

docker build --pull -f "/your/path/to/Dockerfile" -t "img_name:tag" "/your/path/to/workdir"

# Replace 6667 with your port
docker run -p 6667:22  -it -d --name="container_name" img_name:tag

# Test docker ssh
docker ps
ssh -p 6667 root@0.0.0.0