本課主題
- SQLAlchemy - Core
- SQLAlchemy - ORM
- Paramiko 介紹和操作
- 上下文操作应用
- 初探堡垒机
SQLAlchemy - Core
连接 URL
通过 create_engine 方法创建 MySQL 数据库的连接，create_engine("url") 接受一个 URL 连接:
>>> MySQL-Python: mysql+mysqldb://<user>:<password>@<host>[:<port>]/<dbname>
>>> pymysql: mysql+pymysql://<username>:<password>@<host>/<dbname>[?<options>]
>>> MySQL-Connector: mysql+mysqlconnector://<user>:<password>@<host>[:<port>]/<dbname>
>>> cx_Oracle: oracle+cx_oracle://user:pass@host:port/dbname[?key=value&key=value...]
>>> PostgreSQL: postgresql+pg8000://<user>:<password>@<host>/<db>
Details: http://docs.sqlalchemy.org/en/latest/dialects/index.html
create_engine("url")
用 SQLAlchemy 来创建数据库表
from datetime import datetime # ------------------------------------------------------------
# Import the datatype
# ------------------------------------------------------------
from sqlalchemy import (
MetaData, Table, Column, Integer, Numeric, String, Float, Boolean,
DateTime, ForeignKey, create_engine
)
metadata = MetaData() # ------------------------------------------------------------
# Table Objects
# ------------------------------------------------------------
# 创建 cookies 的数据库表
cookies = Table('cookies', metadata,
Column('cookie_id', Integer(), primary_key=True),
Column('cookie_name', String(50), index=True), #创建索引
Column('cookie_recipe_url', String(255)),
Column('cookie_sku', String(55)),
Column('quantity', Integer()),
Column('unit_cost', Numeric(12,2)) # Index('ix_cookies_cookie_name', 'cookie_name') #也可以调用这个方法来创建索引
) # 创建 users 的数据库表
user = Table('users', metadata,
Column('user_id', Integer(), primary_key=True),
Column('username', String(15), nullable=False, unique=True),
Column('email_address', String(255), nullable=False),
Column('phone', String(20), nullable=False),
Column('password', String(25), nullable=False),
Column('created_on', DateTime(), default=datetime.now),
Column('updated_on', DateTime(), default=datetime.now, onupdate=datetime.now) # PrimaryKeyConstraint('user_id', name='user_pk'), #也可以调用这个方法来创建主键
# UniqueConstraint('username', name='uix_username'), #也可以调用这个方法来创建唯一
# CheckConstraint('unit_cost >= 0.00', name='unit_cost_positive')
) # 创建 orders 的数据库表
orders = Table('orders', metadata,
Column('order_id', Integer(), primary_key=True),
Column('user_id', ForeignKey('users.user_id')),
Column('shipped', Boolean(), default=False)
) # 创建 line_items 的数据库表
line_items = Table('line_items', metadata,
Column('line_items_id', Integer(), primary_key=True),
Column('order_id', ForeignKey('orders.order_id')),
Column('cookie_id', ForeignKey('cookies.cookie_id')),
Column('quantity', Integer()),
Column('extended_cost', Numeric(12,2))
) # ------------------------------------------------------------
# 连接url
# ------------------------------------------------------------
engine = create_engine("mysql+pymysql://root@localhost/demo_db", pool_recycle=3600) # ------------------------------------------------------------
# 调用 create_all() 方法来创建所有数据库表
# 默认该方法不会对已经存在的数据库表重新创建。
# ------------------------------------------------------------
metadata.create_all(engine)
metadata.create_all(engine)
SQLAlchemy - ORM
这是一个叫 Object Relational Mapping 框架，可以让我们通过类和对象来操作数据库，具体功能包括创建表，定义数据类型，新增或者查询，一般 MySQL 能做的功能，都可以在 SQLALchemy 里实现，我也是用上一章的那个数据模型去介紹如何用 SQLALchemy 的API去操作数据库。
一对多例子：
从人的角度看：每个人只可以选择一种颜色（一对一）；从颜色的角度看：但是一种颜色可以有多个人选择（一对多）
多对多例子：
从组的角度看：每个组可以有不同的服务器（一对多）；从服务器的角度看：每个服务器也可以属于不同的组（一对多）
表操作
- 创建连接
engine = create_engine('mysql+pymysql://myuser:mypass@192.168.80.128:3306/s13', max_overflow = 5)创建 MySQL连接
- 创建表
Base.metadata.create_all(engine)
Base.metadata.create_all( )
#!/usr/bin/env python
# -*- coding:utf-8 -*- from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, ForeignKey, UniqueConstraint, Index
from sqlalchemy.orm import sessionmaker, relationship
from sqlalchemy import create_engine engine = create_engine('mysql+pymysql://myuser:mypass@192.168.80.128:3306/s13', max_overflow = 5) # 创建连接
Base = declarative_base() # Many-to-many
class Servers_to_Groups(Base):
__tablename__ = 'rel_servers_groups'
nid = Column(Integer, nullable=False, primary_key=True)
server_id = Column(Integer, ForeignKey('servers.id'))
group_id = Column(Integer, ForeignKey('groups.id')) # 在 sqlalchemy 支持创建 relationship，方便查询
groups = relationship('Groups', backref = 'lkp_servers') # 在groups表的內部, sqlalchemy 會多創建一個隱性字段名叫 lkp_servers
servers = relationship('Servers', backref = 'lkp_groups') # 在servers表的內部, sqlalchemy 會多創建一個隱性字段名叫 lkp_groups class Groups(Base):
__tablename__ = 'groups'
id = Column(Integer, nullable=False, primary_key=True)
name = Column(String(50), unique=True, nullable=False)
port = Column(Integer, default=22) class Servers(Base):
__tablename__ = 'servers'
id = Column(Integer, nullable=False, primary_key=True)
name = Column(String(64), unique=True, nullable=False)sqlalchemy创建表(多对多)
#!/usr/bin/env python
# -*- coding:utf-8 -*- from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, ForeignKey, UniqueConstraint, Index
from sqlalchemy.orm import sessionmaker, relationship
from sqlalchemy import create_engine engine = create_engine('mysql+pymysql://myuser:mypass@192.168.80.128:3306/s13', max_overflow = 5) # 创建连接
Base = declarative_base() class Users(Base):
__tablename__ = 'users'
id = Column(Integer, nullable=False, primary_key=True)
name = Column(String(32))
extra = Column(String(16)) __table_args__ = (
UniqueConstraint('id','name', name = 'unix_id_name'),
Index('ix_id_name','name','extra')
) # One-to-many
class Favor(Base):
__tablename__ = 'favor'
nid = Column(Integer, primary_key=True)
caption = Column(String(32), default='red', unique=True) #当打印这个类时,会使用以下方法
def __repr__(self):
return "%s-%s" %(self.nid, self.caption) class Person(Base):
__tablename__ = 'person'
nid = Column(Integer, primary_key=True)
name = Column(String(32), index=True, nullable=True)
favor_id = Column(Integer, ForeignKey('favor.nid')) favor = relationship('favor', backref='lkp_person') # 在favor表的內部,sqlalchemy會多創建一個隱性字段名叫lkp_personsqlalchemy创建表(一对多)
- 删除表 DROP TABLE
Base.metadata.drop_all(engine)
Base.metadata.drop_all( )
- 定义自动增量 AUTO INCREMEN／主键 PRIMARY KEY
sid = Column(Integer, primary_key=True, autoincrement=True)
autoincrement,primary_key
- 定义外键 FOREIGN KEY
person_sid = Column(Integer, ForeignKey("dm_person.sid"),nullable=False)ForeignKey( )
- 定义关系 relationship
class Favor(Base):
__tablename__ = 'favor'
nid = Column(Integer, primary_key=True)
caption = Column(String(32), default='red', unique=True) #当打印这个类时,会使用以下方法
def __repr__(self):
return "%s-%s" %(self.nid, self.caption) class Person(Base):
__tablename__ = 'person'
nid = Column(Integer, primary_key=True)
name = Column(String(32), index=True, nullable=True)
favor_id = Column(Integer, ForeignKey('favor.nid')) favor = relationship('favor', backref='lkp_person')relationship( )
- 定义约束 CONSTRAINT
class Users(Base):
__tablename__ = 'users'
id = Column(Integer, nullable=False, primary_key=True)
name = Column(String(32))
extra = Column(String(16)) __table_args__ = (
UniqueConstraint('id','name', name = 'unix_id_name'),
Index('ix_id_name','name','extra')
)sqlalchemy中的约束
- ALTER TABLE
- WHERE
# WHERE CLAUSE
record = session.query(Users).filter_by(name='janice').all()
record = session.query(Users).filter(Users.id > 1, Users.name == 'janice').all()
record = session.query(Users).filter(Users.id.between(1,3)).all()
record = session.query(Users).filter(Users.id.in_([1,3,4])).all()
record = session.query(Users).filter(~Users.id.in_([1,3,4])).all()
record = session.query(Users).filter(Users.id.in_(session.query(Users.id).filter_by(name='janice'))).all()session.query( ).filter( )
- ORDER BY
record = session.query(Users).order_by(Users.name.desc()).all()
record = session.query(Users).order_by(Users.name.desc(), Users.id.asc()).all() for res in record:
print(res.name, res.extra) """
ken manager
janice engineer
alex director
"""session.query( ).order_by(User.name.desc())
- GROUP BY
record = session.query(
func.max(Users.id),
func.min(Users.id),
func.sum(Users.id)
).group_by(Users.name).having(func.min(Users.id) > 2).all() for res in record:
print(res)session.query().group_by()
- UNION/ UNION ALL
q1 = session.query(Users.name).filter(Users.id > 2)
q2 = session.query(Favor.nid).filter(Favor.nid > 2) # record = q1.union(q2)
record = q1.union_all(q2) for res in record.all():
print(res)q1.union()/union_all(q2)
- LIKE
# record = session.query(Users).filter(Users.name.like('j%')).all()
record = session.query(Users).filter(~Users.name.like('j%')).all() for res in record:
print(res.name, res.extra)session.query().filter(User.name.like())
- LIMIT
record = session.query(Users)[1:2]
session.query()[1:2]
- INSERT
session.add_all([
Favor(caption='red'),
Favor(caption='orange'),
Favor(caption='yellow'),
Favor(caption='green'),
Favor(caption='blue'),
Favor(caption='purple')
]) session.add_all([
Person(name='janice',favor_id=4),
Person(name='alex',favor_id=1),
Person(name='kennith',favor_id=6),
Person(name='peter',favor_id=3),
Person(name='jennifer',favor_id=2),
Person(name='winnie',favor_id=5)
])session.add()/add_all([])
- DELETE
session.query(Users).filter(Users.id > 2).delete()
session.query().delete()
- UPDATE
session.query(Users).filter(Users.id > 2).update({"name":"updated_kennith"})
session.query(Users).filter(Users.id > 2).update({Users.name: Users.name + "_100"}, synchronize_session=False)
session.query(Users).filter(Users.id > 2).update({"id": Users.id + 100}, synchronize_session="evaluate")session.query().update()
- SELECT
record = session.query(Users) #可以查看原生 SQL 語句
record = session.query(Users).all() # 返回的是Users object对象
record = session.query(Users.name, Users.extra).all() # 返回的是Users.name 和 Users.extra 的内容，因为Users.name, Users.extra作为参数传入了 query()
record = session.query(Users).filter_by(name='alex').all() # 返回的是一个可迭代的对象 e.g.for res in record; print(res.name)
record = session.query(Users).filter_by(name='alex').first() # 返回的是Users.name 和 Users.extra 的內容session.query().first()/all()
- INNER JOIN / LEFT OUTER JOIN
record = session.query(Favor, Person).filter(Person.favor_id==Favor.nid).all()
for res in record:
print(res[0], res[1].name) record = session.query(Person.name,Favor.caption).join(Favor).all()
record = session.query(Person).join(Favor, isouter=True)
print(record)session.query().join()/.join(isouter=True)
- SUM/ MIN/ MAX
record = session.query(
func.max(Users.id),
func.min(Users.id),
func.sum(Users.id)
).group_by(Users.name).having(func.min(Users.id) > 2).all() for res in record:
print(res) """
(3, 3, Decimal('3'))
"""SUM/ MIN/ MAX
数据分析例子
现在我会用我上一章的那个数据模型来回答以下问题，这是一个记录了每个用户购买了那些货品的一张数据表，还有记录了购买了多少件数和每件货品的价格是多少。
第一步是在数据库上创建以上的表，分别是产品，人和销售表。产品表记录了一些产品名称和产品类型; 人表记录了购买人的名称和年龄；销售表完整的记录了谁买了什么、买了多少产品和产品的价格是多少、等等...现在会使用 Python 的 SQLAlchemy 去完成所有的工作,而不是用原生 SQL 去做。
- 第一步：创建产品，人和销售表的表
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, ForeignKey, UniqueConstraint, Index
from sqlalchemy.orm import sessionmaker, relationship
from sqlalchemy import create_engine # 创建连接
engine = create_engine('mysql+pymysql://myuser:mypass@172.16.201.134:3306/s13?',
max_overflow = 5,
pool_recycle=3600) Base = declarative_base() class Person(Base):
__tablename__ = 'dm_person'
sid = Column(Integer, primary_key=True, autoincrement=True)
name = Column(String(10), unique=True, nullable=False)
age = Column(Integer, nullable=False) class Product(Base):
__tablename__ = 'dm_product'
sid = Column(Integer, primary_key=True, autoincrement=True)
product_name = Column(String(50), unique=True, nullable=False)
product_category = Column(String(50), nullable=False) class Sales(Base):
__tablename__ = 'fct_sales'
sid = Column(Integer, primary_key=True, autoincrement=True)
person_sid = Column(Integer, ForeignKey("dm_person.sid"),nullable=False)
product_sid = Column(Integer, ForeignKey("dm_product.sid"),nullable=False)
unit_price = Column(Integer, nullable=True)
qty = Column(Integer, nullable=True) person = relationship("Person","lkp_sales")
product = relationship("Product","lkp_sales")创建表ORM模型
- 第二步：创建一个 session 来进行数据操作
Session = sessionmaker(bind=engine)
session = Session()
# blablabla...
session.commit()Session = sessionmaker(bind=engine)
- 第三步：插入一些测试数据，此时,我会调用刚才学的 session.add_all( )方法
def data_init(): # 创建连接
Session = sessionmaker(bind=engine)
session = Session() # 初始化插入数据 - Person Table
session.add_all([
Person(name='janice', age=22),
Person(name='alex', age=33),
Person(name='ken', age=30),
Person(name='peter', age=28),
Person(name='david', age=23),
Person(name='ziv', age=25),
Person(name='ronald', age=21),
Person(name='kenny', age=36)
]) # 初始化插入数据 - Product Table
session.add_all([
Product(product_name='iPhone 6S', product_category='Electronic'),
Product(product_name='iPhone 7', product_category='Electronic'),
Product(product_name='XiaoMi 5', product_category='Electronic'),
Product(product_name='Samsung Note 7', product_category='Electronic'),
Product(product_name='Programming in Python', product_category='Books'),
Product(product_name='Python In Action', product_category='Books'),
Product(product_name='Shakespeare', product_category='Books'),
Product(product_name='Coconut Water', product_category='Foods and Drinks'),
Product(product_name='Coffe', product_category='Foods and Drinks'),
Product(product_name='Bike', product_category='Automobile'),
Product(product_name='Tesla Model X', product_category='Automobile')
]) # SQL commit
session.commit() data_init()session.add( )/ add_all( )函数例子
数据准备好了，现在可以尝试回答问题啦！
-
问题一：我想知道这商店有什么产品，可以调用 session.query( ) 去查询
print("问题一：我想知道这商店有什么产品?")
Session = sessionmaker(bind=engine)
session = Session() record = session.query(Product.product_name, Product.product_category).all() print("\n答: 这商店有以下产品:")
for row in enumerate(record,1):
print('{}. {}'.format(row[0],row[1][0])) # record = session.query(Product).all()
# for r in enumerate(record, 1):
# print('{}. {}'.format(r[0],r[1].product_name)) session.commit() """
问题一：我想知道这商店有什么产品? 答: 这商店有以下产品:
1. iPhone 6S
2. iPhone 7
3. XiaoMi 5
4. Samsung Note 7
5. Programming in Python
6. Python In Action
7. Shakespeare
8. Coconut Water
9. Coffe
10. Bike
11. Tesla Model X
"""session.query( )函数例子
-
问题二：我想知道这商店有什么"电子产品"可以卖 (提示：Electronic)
print("问题二：我想知道这商店有什么'电子产品'可以卖")
Session = sessionmaker(bind=engine)
session = Session() # record = session.query(Product).filter_by(product_category = 'Electronic').all()
record = session.query(Product).filter(Product.product_category == 'Electronic').all() print("\n答: 这商店有以下电子产品:")
for row in enumerate(record,1):
print('{}. {}'.format(row[0],row[1].product_name)) session.commit() """
问题二：我想知道这商店有什么'电子产品'可以卖 答: 这商店有以下电子产品:
1. iPhone 6S
2. iPhone 7
3. XiaoMi 5
4. Samsung Note 7
"""session.query( ).filter( )/ filter_by( )函数例子
-
问题三：我想知道 janice 买了什么东西
print("问题三：我想知道 janice 买了什么东西")
Session = sessionmaker(bind=engine)
session = Session() #method1
record = session.query(Product.product_name).filter(Sales.person_sid==Person.sid, Sales.product_sid==Product.sid, Person.name == 'janice').all() print("\n答: janice 买了以下东西:")
for row in enumerate(record,1):
print('{}. {}'.format(row[0],row[1].product_name)) session.commit() """
问题三：我想知道 janice 买了什么东西 答: janice 买了以下东西:
1. Shakespeare
2. Coffe
3. Samsung Note 7
4. Tesla Model X
5. iPhone 7
6. XiaoMi 5
7. Coconut Water
8. Python In Action
"""表关联和filter( )函数例子
-
问题四：我想知道 janice 总共花费了多少钱
print("问题四：我想知道 janice 总共花费了多少钱")
Session = sessionmaker(bind=engine)
session = Session() from sqlalchemy.sql import func record = session.query(
func.sum(Sales.unit_price * Sales.qty)
).filter(Sales.person_sid==Person.sid,
Sales.product_sid==Product.sid,
Person.name == 'janice').group_by(Person.name).all() print("\n答: janice 总共花费了 ${}".format(record[0][0])) session.commit() """
问题四：我想知道 janice 总共花费了多少钱
答: janice 总共花费了 $880595 """表关联, sum(), group_by()和filter( )函数例子
-
问题五：我想知道每个用户总共花费了多少钱，以花费最多的排序
print("问题五：我想知道每个用户总共花费了多少钱，以花费最多的排序")
Session = sessionmaker(bind=engine)
session = Session() from sqlalchemy.sql import func record = session.query(
Person.name,
func.sum(Sales.unit_price * Sales.qty)
).filter(Sales.person_sid==Person.sid,
Sales.product_sid==Product.sid).group_by(Person.name).order_by(func.sum(Sales.unit_price * Sales.qty).desc()).all() for row in enumerate(record,1):
print('第{}名: {}，总共花费了 ${}元'.format(row[0],row[1][0],row[1][1])) session.commit() """
问题五：我想知道每个用户总共花费了多少钱，以花费最多的排序 第1名: janice，总共花费了 $880595元
第2名: peter，总共花费了 $820101元
第3名: alex，总共花费了 $97684元
第4名: ken，总共花费了 $97126元
"""表关联, sum(), group_by(), order_by()和filter( )函数例子
-
问题六：我想知道谁买了 Tesla 又买了一个小米手机
print("问题六：我想知道谁买了 Tesla 又买了一个小米手机")
Session = sessionmaker(bind=engine)
session = Session() from sqlalchemy import and_, or_, distinct record = session.query(distinct(Person.name,)).\
filter(Sales.person_sid==Person.sid,Sales.product_sid==Product.sid).\
filter(Product.product_name.in_(['Tesla Model X','XiaoMi 5'])).all() print("买了 Tesla 又买了一个小米手机是:")
ret = [row[0] for row in record]
print(ret) """
问题六：我想知道谁买了 Tesla 又买了一个小米手机
买了 Tesla 又买了一个小米手机是:
['janice', 'peter', 'ken']
"""表关联,in_(), distinct()
Paramiko 介紹和操作
paramiko模块，基于SSH用于连接远程服务器并执行相关操作，先给大家有一个大概的概念，Paramiko 可支持以下两种登入方法和创建三种不同的对象来完成工作:
- 创建 SSHClient e.g. paramiko.SSHClient( ) - 用于连接远程服务器并执行基本命令
class SSHClient (ClosingContextManager):
"""
A high-level representation of a session with an SSH server. This class
wraps `.Transport`, `.Channel`, and `.SFTPClient` to take care of most
aspects of authenticating and opening channels. A typical use case is:: client = SSHClient()
client.load_system_host_keys()
client.connect('ssh.example.com')
stdin, stdout, stderr = client.exec_command('ls -l') You may pass in explicit overrides for authentication and server host key
checking. The default mechanism is to try to use local key files or an
SSH agent (if one is running). Instances of this class may be used as context managers. .. versionadded:: 1.6
""" def __init__(self):
"""
Create a new SSHClient.
"""
self._system_host_keys = HostKeys()
self._host_keys = HostKeys()
self._host_keys_filename = None
self._log_channel = None
self._policy = RejectPolicy()
self._transport = None
self._agent = None def load_system_host_keys(self, filename=None):
"""
Load host keys from a system (read-only) file. Host keys read with
this method will not be saved back by `save_host_keys`. This method can be called multiple times. Each new set of host keys
will be merged with the existing set (new replacing old if there are
conflicts). If ``filename`` is left as ``None``, an attempt will be made to read
keys from the user's local "known hosts" file, as used by OpenSSH,
and no exception will be raised if the file can't be read. This is
probably only useful on posix. :param str filename: the filename to read, or ``None`` :raises IOError:
if a filename was provided and the file could not be read
"""
if filename is None:
# try the user's .ssh key file, and mask exceptions
filename = os.path.expanduser('~/.ssh/known_hosts')
try:
self._system_host_keys.load(filename)
except IOError:
pass
return
self._system_host_keys.load(filename) def load_host_keys(self, filename):
"""
Load host keys from a local host-key file. Host keys read with this
method will be checked after keys loaded via `load_system_host_keys`,
but will be saved back by `save_host_keys` (so they can be modified).
The missing host key policy `.AutoAddPolicy` adds keys to this set and
saves them, when connecting to a previously-unknown server. This method can be called multiple times. Each new set of host keys
will be merged with the existing set (new replacing old if there are
conflicts). When automatically saving, the last hostname is used. :param str filename: the filename to read :raises IOError: if the filename could not be read
"""
self._host_keys_filename = filename
self._host_keys.load(filename) def save_host_keys(self, filename):
"""
Save the host keys back to a file. Only the host keys loaded with
`load_host_keys` (plus any added directly) will be saved -- not any
host keys loaded with `load_system_host_keys`. :param str filename: the filename to save to :raises IOError: if the file could not be written
""" # update local host keys from file (in case other SSH clients
# have written to the known_hosts file meanwhile.
if self._host_keys_filename is not None:
self.load_host_keys(self._host_keys_filename) with open(filename, 'w') as f:
for hostname, keys in self._host_keys.items():
for keytype, key in keys.items():
f.write('%s %s %s\n' % (hostname, keytype, key.get_base64())) def get_host_keys(self):
"""
Get the local `.HostKeys` object. This can be used to examine the
local host keys or change them. :return: the local host keys as a `.HostKeys` object.
"""
return self._host_keys def set_log_channel(self, name):
"""
Set the channel for logging. The default is ``"paramiko.transport"``
but it can be set to anything you want. :param str name: new channel name for logging
"""
self._log_channel = name def set_missing_host_key_policy(self, policy):
"""
Set policy to use when connecting to servers without a known host key. Specifically: * A **policy** is an instance of a "policy class", namely some subclass
of `.MissingHostKeyPolicy` such as `.RejectPolicy` (the default),
`.AutoAddPolicy`, `.WarningPolicy`, or a user-created subclass. .. note::
This method takes class **instances**, not **classes** themselves.
Thus it must be called as e.g.
``.set_missing_host_key_policy(WarningPolicy())`` and *not*
``.set_missing_host_key_policy(WarningPolicy)``. * A host key is **known** when it appears in the client object's cached
host keys structures (those manipulated by `load_system_host_keys`
and/or `load_host_keys`). :param .MissingHostKeyPolicy policy:
the policy to use when receiving a host key from a
previously-unknown server
"""
self._policy = policy def _families_and_addresses(self, hostname, port):
"""
Yield pairs of address families and addresses to try for connecting. :param str hostname: the server to connect to
:param int port: the server port to connect to
:returns: Yields an iterable of ``(family, address)`` tuples
"""
guess = True
addrinfos = socket.getaddrinfo(hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
for (family, socktype, proto, canonname, sockaddr) in addrinfos:
if socktype == socket.SOCK_STREAM:
yield family, sockaddr
guess = False # some OS like AIX don't indicate SOCK_STREAM support, so just guess. :(
# We only do this if we did not get a single result marked as socktype == SOCK_STREAM.
if guess:
for family, _, _, _, sockaddr in addrinfos:
yield family, sockaddr def connect(
self,
hostname,
port=SSH_PORT,
username=None,
password=None,
pkey=None,
key_filename=None,
timeout=None,
allow_agent=True,
look_for_keys=True,
compress=False,
sock=None,
gss_auth=False,
gss_kex=False,
gss_deleg_creds=True,
gss_host=None,
banner_timeout=None
):
"""
Connect to an SSH server and authenticate to it. The server's host key
is checked against the system host keys (see `load_system_host_keys`)
and any local host keys (`load_host_keys`). If the server's hostname
is not found in either set of host keys, the missing host key policy
is used (see `set_missing_host_key_policy`). The default policy is
to reject the key and raise an `.SSHException`. Authentication is attempted in the following order of priority: - The ``pkey`` or ``key_filename`` passed in (if any)
- Any key we can find through an SSH agent
- Any "id_rsa", "id_dsa" or "id_ecdsa" key discoverable in
``~/.ssh/``
- Plain username/password auth, if a password was given If a private key requires a password to unlock it, and a password is
passed in, that password will be used to attempt to unlock the key. :param str hostname: the server to connect to
:param int port: the server port to connect to
:param str username:
the username to authenticate as (defaults to the current local
username)
:param str password:
a password to use for authentication or for unlocking a private key
:param .PKey pkey: an optional private key to use for authentication
:param str key_filename:
the filename, or list of filenames, of optional private key(s) to
try for authentication
:param float timeout:
an optional timeout (in seconds) for the TCP connect
:param bool allow_agent:
set to False to disable connecting to the SSH agent
:param bool look_for_keys:
set to False to disable searching for discoverable private key
files in ``~/.ssh/``
:param bool compress: set to True to turn on compression
:param socket sock:
an open socket or socket-like object (such as a `.Channel`) to use
for communication to the target host
:param bool gss_auth:
``True`` if you want to use GSS-API authentication
:param bool gss_kex:
Perform GSS-API Key Exchange and user authentication
:param bool gss_deleg_creds: Delegate GSS-API client credentials or not
:param str gss_host:
The targets name in the kerberos database. default: hostname
:param float banner_timeout: an optional timeout (in seconds) to wait
for the SSH banner to be presented. :raises BadHostKeyException: if the server's host key could not be
verified
:raises AuthenticationException: if authentication failed
:raises SSHException: if there was any other error connecting or
establishing an SSH session
:raises socket.error: if a socket error occurred while connecting .. versionchanged:: 1.15
Added the ``banner_timeout``, ``gss_auth``, ``gss_kex``,
``gss_deleg_creds`` and ``gss_host`` arguments.
"""
if not sock:
errors = {}
# Try multiple possible address families (e.g. IPv4 vs IPv6)
to_try = list(self._families_and_addresses(hostname, port))
for af, addr in to_try:
try:
sock = socket.socket(af, socket.SOCK_STREAM)
if timeout is not None:
try:
sock.settimeout(timeout)
except:
pass
retry_on_signal(lambda: sock.connect(addr))
# Break out of the loop on success
break
except socket.error as e:
# Raise anything that isn't a straight up connection error
# (such as a resolution error)
if e.errno not in (ECONNREFUSED, EHOSTUNREACH):
raise
# Capture anything else so we know how the run looks once
# iteration is complete. Retain info about which attempt
# this was.
errors[addr] = e # Make sure we explode usefully if no address family attempts
# succeeded. We've no way of knowing which error is the "right"
# one, so we construct a hybrid exception containing all the real
# ones, of a subclass that client code should still be watching for
# (socket.error)
if len(errors) == len(to_try):
raise NoValidConnectionsError(errors) t = self._transport = Transport(sock, gss_kex=gss_kex, gss_deleg_creds=gss_deleg_creds)
t.use_compression(compress=compress)
if gss_kex and gss_host is None:
t.set_gss_host(hostname)
elif gss_kex and gss_host is not None:
t.set_gss_host(gss_host)
else:
pass
if self._log_channel is not None:
t.set_log_channel(self._log_channel)
if banner_timeout is not None:
t.banner_timeout = banner_timeout
t.start_client()
ResourceManager.register(self, t) server_key = t.get_remote_server_key()
keytype = server_key.get_name() if port == SSH_PORT:
server_hostkey_name = hostname
else:
server_hostkey_name = "[%s]:%d" % (hostname, port) # If GSS-API Key Exchange is performed we are not required to check the
# host key, because the host is authenticated via GSS-API / SSPI as
# well as our client.
if not self._transport.use_gss_kex:
our_server_key = self._system_host_keys.get(server_hostkey_name,
{}).get(keytype, None)
if our_server_key is None:
our_server_key = self._host_keys.get(server_hostkey_name,
{}).get(keytype, None)
if our_server_key is None:
# will raise exception if the key is rejected; let that fall out
self._policy.missing_host_key(self, server_hostkey_name,
server_key)
# if the callback returns, assume the key is ok
our_server_key = server_key if server_key != our_server_key:
raise BadHostKeyException(hostname, server_key, our_server_key) if username is None:
username = getpass.getuser() if key_filename is None:
key_filenames = []
elif isinstance(key_filename, string_types):
key_filenames = [key_filename]
else:
key_filenames = key_filename
if gss_host is None:
gss_host = hostname
self._auth(username, password, pkey, key_filenames, allow_agent,
look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host) def close(self):
"""
Close this SSHClient and its underlying `.Transport`. .. warning::
Failure to do this may, in some situations, cause your Python
interpreter to hang at shutdown (often due to race conditions).
It's good practice to `close` your client objects anytime you're
done using them, instead of relying on garbage collection.
"""
if self._transport is None:
return
self._transport.close()
self._transport = None if self._agent is not None:
self._agent.close()
self._agent = None def exec_command(self, command, bufsize=-1, timeout=None, get_pty=False):
"""
Execute a command on the SSH server. A new `.Channel` is opened and
the requested command is executed. The command's input and output
streams are returned as Python ``file``-like objects representing
stdin, stdout, and stderr. :param str command: the command to execute
:param int bufsize:
interpreted the same way as by the built-in ``file()`` function in
Python
:param int timeout:
set command's channel timeout. See `Channel.settimeout`.settimeout
:return:
the stdin, stdout, and stderr of the executing command, as a
3-tuple :raises SSHException: if the server fails to execute the command
"""
chan = self._transport.open_session(timeout=timeout)
if get_pty:
chan.get_pty()
chan.settimeout(timeout)
chan.exec_command(command)
stdin = chan.makefile('wb', bufsize)
stdout = chan.makefile('r', bufsize)
stderr = chan.makefile_stderr('r', bufsize)
return stdin, stdout, stderr def invoke_shell(self, term='vt100', width=80, height=24, width_pixels=0,
height_pixels=0):
"""
Start an interactive shell session on the SSH server. A new `.Channel`
is opened and connected to a pseudo-terminal using the requested
terminal type and size. :param str term:
the terminal type to emulate (for example, ``"vt100"``)
:param int width: the width (in characters) of the terminal window
:param int height: the height (in characters) of the terminal window
:param int width_pixels: the width (in pixels) of the terminal window
:param int height_pixels: the height (in pixels) of the terminal window
:return: a new `.Channel` connected to the remote shell :raises SSHException: if the server fails to invoke a shell
"""
chan = self._transport.open_session()
chan.get_pty(term, width, height, width_pixels, height_pixels)
chan.invoke_shell()
return chan def open_sftp(self):
"""
Open an SFTP session on the SSH server. :return: a new `.SFTPClient` session object
"""
return self._transport.open_sftp_client() def get_transport(self):
"""
Return the underlying `.Transport` object for this SSH connection.
This can be used to perform lower-level tasks, like opening specific
kinds of channels. :return: the `.Transport` for this connection
"""
return self._transport def _auth(self, username, password, pkey, key_filenames, allow_agent,
look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host):
"""
Try, in order: - The key passed in, if one was passed in.
- Any key we can find through an SSH agent (if allowed).
- Any "id_rsa", "id_dsa" or "id_ecdsa" key discoverable in ~/.ssh/
(if allowed).
- Plain username/password auth, if a password was given. (The password might be needed to unlock a private key, or for
two-factor authentication [for which it is required].)
"""
saved_exception = None
two_factor = False
allowed_types = set()
two_factor_types = set(['keyboard-interactive','password']) # If GSS-API support and GSS-PI Key Exchange was performed, we attempt
# authentication with gssapi-keyex.
if gss_kex and self._transport.gss_kex_used:
try:
self._transport.auth_gssapi_keyex(username)
return
except Exception as e:
saved_exception = e # Try GSS-API authentication (gssapi-with-mic) only if GSS-API Key
# Exchange is not performed, because if we use GSS-API for the key
# exchange, there is already a fully established GSS-API context, so
# why should we do that again?
if gss_auth:
try:
self._transport.auth_gssapi_with_mic(username, gss_host,
gss_deleg_creds)
return
except Exception as e:
saved_exception = e if pkey is not None:
try:
self._log(DEBUG, 'Trying SSH key %s' % hexlify(pkey.get_fingerprint()))
allowed_types = set(self._transport.auth_publickey(username, pkey))
two_factor = (allowed_types & two_factor_types)
if not two_factor:
return
except SSHException as e:
saved_exception = e if not two_factor:
for key_filename in key_filenames:
for pkey_class in (RSAKey, DSSKey, ECDSAKey):
try:
key = pkey_class.from_private_key_file(key_filename, password)
self._log(DEBUG, 'Trying key %s from %s' % (hexlify(key.get_fingerprint()), key_filename))
allowed_types = set(self._transport.auth_publickey(username, key))
two_factor = (allowed_types & two_factor_types)
if not two_factor:
return
break
except SSHException as e:
saved_exception = e if not two_factor and allow_agent:
if self._agent is None:
self._agent = Agent() for key in self._agent.get_keys():
try:
self._log(DEBUG, 'Trying SSH agent key %s' % hexlify(key.get_fingerprint()))
# for 2-factor auth a successfully auth'd key password will return an allowed 2fac auth method
allowed_types = set(self._transport.auth_publickey(username, key))
two_factor = (allowed_types & two_factor_types)
if not two_factor:
return
break
except SSHException as e:
saved_exception = e if not two_factor:
keyfiles = []
rsa_key = os.path.expanduser('~/.ssh/id_rsa')
dsa_key = os.path.expanduser('~/.ssh/id_dsa')
ecdsa_key = os.path.expanduser('~/.ssh/id_ecdsa')
if os.path.isfile(rsa_key):
keyfiles.append((RSAKey, rsa_key))
if os.path.isfile(dsa_key):
keyfiles.append((DSSKey, dsa_key))
if os.path.isfile(ecdsa_key):
keyfiles.append((ECDSAKey, ecdsa_key))
# look in ~/ssh/ for windows users:
rsa_key = os.path.expanduser('~/ssh/id_rsa')
dsa_key = os.path.expanduser('~/ssh/id_dsa')
ecdsa_key = os.path.expanduser('~/ssh/id_ecdsa')
if os.path.isfile(rsa_key):
keyfiles.append((RSAKey, rsa_key))
if os.path.isfile(dsa_key):
keyfiles.append((DSSKey, dsa_key))
if os.path.isfile(ecdsa_key):
keyfiles.append((ECDSAKey, ecdsa_key)) if not look_for_keys:
keyfiles = [] for pkey_class, filename in keyfiles:
try:
key = pkey_class.from_private_key_file(filename, password)
self._log(DEBUG, 'Trying discovered key %s in %s' % (hexlify(key.get_fingerprint()), filename))
# for 2-factor auth a successfully auth'd key will result in ['password']
allowed_types = set(self._transport.auth_publickey(username, key))
two_factor = (allowed_types & two_factor_types)
if not two_factor:
return
break
except (SSHException, IOError) as e:
saved_exception = e if password is not None:
try:
self._transport.auth_password(username, password)
return
except SSHException as e:
saved_exception = e
elif two_factor:
try:
self._transport.auth_interactive_dumb(username)
return
except SSHException as e:
saved_exception = e # if we got an auth-failed exception earlier, re-raise it
if saved_exception is not None:
raise saved_exception
raise SSHException('No authentication methods available') def _log(self, level, msg):
self._transport._log(level, msg)class SSHClient( )源码
- 基于用户名密码连接
- 基于公钥密钥连接
- 创建 SFTPClient e.g. paramiko.SFTPClient.from_transport(transport) - 用于连接远程服务器并执行上传下载
class SFTPClient(BaseSFTP, ClosingContextManager):
"""
SFTP client object. Used to open an SFTP session across an open SSH `.Transport` and perform
remote file operations. Instances of this class may be used as context managers.
"""
def __init__(self, sock):
"""
Create an SFTP client from an existing `.Channel`. The channel
should already have requested the ``"sftp"`` subsystem. An alternate way to create an SFTP client context is by using
`from_transport`. :param .Channel sock: an open `.Channel` using the ``"sftp"`` subsystem :raises SSHException: if there's an exception while negotiating
sftp
"""
BaseSFTP.__init__(self)
self.sock = sock
self.ultra_debug = False
self.request_number = 1
# lock for request_number
self._lock = threading.Lock()
self._cwd = None
# request # -> SFTPFile
self._expecting = weakref.WeakValueDictionary()
if type(sock) is Channel:
# override default logger
transport = self.sock.get_transport()
self.logger = util.get_logger(transport.get_log_channel() + '.sftp')
self.ultra_debug = transport.get_hexdump()
try:
server_version = self._send_version()
except EOFError:
raise SSHException('EOF during negotiation')
self._log(INFO, 'Opened sftp connection (server version %d)' % server_version) @classmethod
def from_transport(cls, t, window_size=None, max_packet_size=None):
"""
Create an SFTP client channel from an open `.Transport`. Setting the window and packet sizes might affect the transfer speed.
The default settings in the `.Transport` class are the same as in
OpenSSH and should work adequately for both files transfers and
interactive sessions. :param .Transport t: an open `.Transport` which is already authenticated
:param int window_size:
optional window size for the `.SFTPClient` session.
:param int max_packet_size:
optional max packet size for the `.SFTPClient` session.. :return:
a new `.SFTPClient` object, referring to an sftp session (channel)
across the transport .. versionchanged:: 1.15
Added the ``window_size`` and ``max_packet_size`` arguments.
"""
chan = t.open_session(window_size=window_size,
max_packet_size=max_packet_size)
if chan is None:
return None
chan.invoke_subsystem('sftp')
return cls(chan) def _log(self, level, msg, *args):
if isinstance(msg, list):
for m in msg:
self._log(level, m, *args)
else:
# escape '%' in msg (they could come from file or directory names) before logging
msg = msg.replace('%','%%')
super(SFTPClient, self)._log(level, "[chan %s] " + msg, *([self.sock.get_name()] + list(args))) def close(self):
"""
Close the SFTP session and its underlying channel. .. versionadded:: 1.4
"""
self._log(INFO, 'sftp session closed.')
self.sock.close() def get_channel(self):
"""
Return the underlying `.Channel` object for this SFTP session. This
might be useful for doing things like setting a timeout on the channel. .. versionadded:: 1.7.1
"""
return self.sock def listdir(self, path='.'):
"""
Return a list containing the names of the entries in the given ``path``. The list is in arbitrary order. It does not include the special
entries ``'.'`` and ``'..'`` even if they are present in the folder.
This method is meant to mirror ``os.listdir`` as closely as possible.
For a list of full `.SFTPAttributes` objects, see `listdir_attr`. :param str path: path to list (defaults to ``'.'``)
"""
return [f.filename for f in self.listdir_attr(path)] def listdir_attr(self, path='.'):
"""
Return a list containing `.SFTPAttributes` objects corresponding to
files in the given ``path``. The list is in arbitrary order. It does
not include the special entries ``'.'`` and ``'..'`` even if they are
present in the folder. The returned `.SFTPAttributes` objects will each have an additional
field: ``longname``, which may contain a formatted string of the file's
attributes, in unix format. The content of this string will probably
depend on the SFTP server implementation. :param str path: path to list (defaults to ``'.'``)
:return: list of `.SFTPAttributes` objects .. versionadded:: 1.2
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'listdir(%r)' % path)
t, msg = self._request(CMD_OPENDIR, path)
if t != CMD_HANDLE:
raise SFTPError('Expected handle')
handle = msg.get_binary()
filelist = []
while True:
try:
t, msg = self._request(CMD_READDIR, handle)
except EOFError:
# done with handle
break
if t != CMD_NAME:
raise SFTPError('Expected name response')
count = msg.get_int()
for i in range(count):
filename = msg.get_text()
longname = msg.get_text()
attr = SFTPAttributes._from_msg(msg, filename, longname)
if (filename != '.') and (filename != '..'):
filelist.append(attr)
self._request(CMD_CLOSE, handle)
return filelist def listdir_iter(self, path='.', read_aheads=50):
"""
Generator version of `.listdir_attr`. See the API docs for `.listdir_attr` for overall details. This function adds one more kwarg on top of `.listdir_attr`:
``read_aheads``, an integer controlling how many
``SSH_FXP_READDIR`` requests are made to the server. The default of 50
should suffice for most file listings as each request/response cycle
may contain multiple files (dependant on server implementation.) .. versionadded:: 1.15
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'listdir(%r)' % path)
t, msg = self._request(CMD_OPENDIR, path) if t != CMD_HANDLE:
raise SFTPError('Expected handle') handle = msg.get_string() nums = list()
while True:
try:
# Send out a bunch of readdir requests so that we can read the
# responses later on Section 6.7 of the SSH file transfer RFC
# explains this
# http://filezilla-project.org/specs/draft-ietf-secsh-filexfer-02.txt
for i in range(read_aheads):
num = self._async_request(type(None), CMD_READDIR, handle)
nums.append(num) # For each of our sent requests
# Read and parse the corresponding packets
# If we're at the end of our queued requests, then fire off
# some more requests
# Exit the loop when we've reached the end of the directory
# handle
for num in nums:
t, pkt_data = self._read_packet()
msg = Message(pkt_data)
new_num = msg.get_int()
if num == new_num:
if t == CMD_STATUS:
self._convert_status(msg)
count = msg.get_int()
for i in range(count):
filename = msg.get_text()
longname = msg.get_text()
attr = SFTPAttributes._from_msg(
msg, filename, longname)
if (filename != '.') and (filename != '..'):
yield attr # If we've hit the end of our queued requests, reset nums.
nums = list() except EOFError:
self._request(CMD_CLOSE, handle)
return def open(self, filename, mode='r', bufsize=-1):
"""
Open a file on the remote server. The arguments are the same as for
Python's built-in `python:file` (aka `python:open`). A file-like
object is returned, which closely mimics the behavior of a normal
Python file object, including the ability to be used as a context
manager. The mode indicates how the file is to be opened: ``'r'`` for reading,
``'w'`` for writing (truncating an existing file), ``'a'`` for
appending, ``'r+'`` for reading/writing, ``'w+'`` for reading/writing
(truncating an existing file), ``'a+'`` for reading/appending. The
Python ``'b'`` flag is ignored, since SSH treats all files as binary.
The ``'U'`` flag is supported in a compatible way. Since 1.5.2, an ``'x'`` flag indicates that the operation should only
succeed if the file was created and did not previously exist. This has
no direct mapping to Python's file flags, but is commonly known as the
``O_EXCL`` flag in posix. The file will be buffered in standard Python style by default, but
can be altered with the ``bufsize`` parameter. ``0`` turns off
buffering, ``1`` uses line buffering, and any number greater than 1
(``>1``) uses that specific buffer size. :param str filename: name of the file to open
:param str mode: mode (Python-style) to open in
:param int bufsize: desired buffering (-1 = default buffer size)
:return: an `.SFTPFile` object representing the open file :raises IOError: if the file could not be opened.
"""
filename = self._adjust_cwd(filename)
self._log(DEBUG, 'open(%r, %r)' % (filename, mode))
imode = 0
if ('r' in mode) or ('+' in mode):
imode |= SFTP_FLAG_READ
if ('w' in mode) or ('+' in mode) or ('a' in mode):
imode |= SFTP_FLAG_WRITE
if 'w' in mode:
imode |= SFTP_FLAG_CREATE | SFTP_FLAG_TRUNC
if 'a' in mode:
imode |= SFTP_FLAG_CREATE | SFTP_FLAG_APPEND
if 'x' in mode:
imode |= SFTP_FLAG_CREATE | SFTP_FLAG_EXCL
attrblock = SFTPAttributes()
t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
if t != CMD_HANDLE:
raise SFTPError('Expected handle')
handle = msg.get_binary()
self._log(DEBUG, 'open(%r, %r) -> %s' % (filename, mode, hexlify(handle)))
return SFTPFile(self, handle, mode, bufsize) # Python continues to vacillate about "open" vs "file"...
file = open def remove(self, path):
"""
Remove the file at the given path. This only works on files; for
removing folders (directories), use `rmdir`. :param str path: path (absolute or relative) of the file to remove :raises IOError: if the path refers to a folder (directory)
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'remove(%r)' % path)
self._request(CMD_REMOVE, path) unlink = remove def rename(self, oldpath, newpath):
"""
Rename a file or folder from ``oldpath`` to ``newpath``. :param str oldpath: existing name of the file or folder
:param str newpath: new name for the file or folder :raises IOError: if ``newpath`` is a folder, or something else goes
wrong
"""
oldpath = self._adjust_cwd(oldpath)
newpath = self._adjust_cwd(newpath)
self._log(DEBUG, 'rename(%r, %r)' % (oldpath, newpath))
self._request(CMD_RENAME, oldpath, newpath) def mkdir(self, path, mode=o777):
"""
Create a folder (directory) named ``path`` with numeric mode ``mode``.
The default mode is 0777 (octal). On some systems, mode is ignored.
Where it is used, the current umask value is first masked out. :param str path: name of the folder to create
:param int mode: permissions (posix-style) for the newly-created folder
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'mkdir(%r, %r)' % (path, mode))
attr = SFTPAttributes()
attr.st_mode = mode
self._request(CMD_MKDIR, path, attr) def rmdir(self, path):
"""
Remove the folder named ``path``. :param str path: name of the folder to remove
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'rmdir(%r)' % path)
self._request(CMD_RMDIR, path) def stat(self, path):
"""
Retrieve information about a file on the remote system. The return
value is an object whose attributes correspond to the attributes of
Python's ``stat`` structure as returned by ``os.stat``, except that it
contains fewer fields. An SFTP server may return as much or as little
info as it wants, so the results may vary from server to server. Unlike a Python `python:stat` object, the result may not be accessed as
a tuple. This is mostly due to the author's slack factor. The fields supported are: ``st_mode``, ``st_size``, ``st_uid``,
``st_gid``, ``st_atime``, and ``st_mtime``. :param str path: the filename to stat
:return:
an `.SFTPAttributes` object containing attributes about the given
file
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'stat(%r)' % path)
t, msg = self._request(CMD_STAT, path)
if t != CMD_ATTRS:
raise SFTPError('Expected attributes')
return SFTPAttributes._from_msg(msg) def lstat(self, path):
"""
Retrieve information about a file on the remote system, without
following symbolic links (shortcuts). This otherwise behaves exactly
the same as `stat`. :param str path: the filename to stat
:return:
an `.SFTPAttributes` object containing attributes about the given
file
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'lstat(%r)' % path)
t, msg = self._request(CMD_LSTAT, path)
if t != CMD_ATTRS:
raise SFTPError('Expected attributes')
return SFTPAttributes._from_msg(msg) def symlink(self, source, dest):
"""
Create a symbolic link (shortcut) of the ``source`` path at
``destination``. :param str source: path of the original file
:param str dest: path of the newly created symlink
"""
dest = self._adjust_cwd(dest)
self._log(DEBUG, 'symlink(%r, %r)' % (source, dest))
source = bytestring(source)
self._request(CMD_SYMLINK, source, dest) def chmod(self, path, mode):
"""
Change the mode (permissions) of a file. The permissions are
unix-style and identical to those used by Python's `os.chmod`
function. :param str path: path of the file to change the permissions of
:param int mode: new permissions
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'chmod(%r, %r)' % (path, mode))
attr = SFTPAttributes()
attr.st_mode = mode
self._request(CMD_SETSTAT, path, attr) def chown(self, path, uid, gid):
"""
Change the owner (``uid``) and group (``gid``) of a file. As with
Python's `os.chown` function, you must pass both arguments, so if you
only want to change one, use `stat` first to retrieve the current
owner and group. :param str path: path of the file to change the owner and group of
:param int uid: new owner's uid
:param int gid: new group id
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'chown(%r, %r, %r)' % (path, uid, gid))
attr = SFTPAttributes()
attr.st_uid, attr.st_gid = uid, gid
self._request(CMD_SETSTAT, path, attr) def utime(self, path, times):
"""
Set the access and modified times of the file specified by ``path``. If
``times`` is ``None``, then the file's access and modified times are set
to the current time. Otherwise, ``times`` must be a 2-tuple of numbers,
of the form ``(atime, mtime)``, which is used to set the access and
modified times, respectively. This bizarre API is mimicked from Python
for the sake of consistency -- I apologize. :param str path: path of the file to modify
:param tuple times:
``None`` or a tuple of (access time, modified time) in standard
internet epoch time (seconds since 01 January 1970 GMT)
"""
path = self._adjust_cwd(path)
if times is None:
times = (time.time(), time.time())
self._log(DEBUG, 'utime(%r, %r)' % (path, times))
attr = SFTPAttributes()
attr.st_atime, attr.st_mtime = times
self._request(CMD_SETSTAT, path, attr) def truncate(self, path, size):
"""
Change the size of the file specified by ``path``. This usually
extends or shrinks the size of the file, just like the `~file.truncate`
method on Python file objects. :param str path: path of the file to modify
:param size: the new size of the file
:type size: int or long
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'truncate(%r, %r)' % (path, size))
attr = SFTPAttributes()
attr.st_size = size
self._request(CMD_SETSTAT, path, attr) def readlink(self, path):
"""
Return the target of a symbolic link (shortcut). You can use
`symlink` to create these. The result may be either an absolute or
relative pathname. :param str path: path of the symbolic link file
:return: target path, as a `str`
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'readlink(%r)' % path)
t, msg = self._request(CMD_READLINK, path)
if t != CMD_NAME:
raise SFTPError('Expected name response')
count = msg.get_int()
if count == 0:
return None
if count != 1:
raise SFTPError('Readlink returned %d results' % count)
return _to_unicode(msg.get_string()) def normalize(self, path):
"""
Return the normalized path (on the server) of a given path. This
can be used to quickly resolve symbolic links or determine what the
server is considering to be the "current folder" (by passing ``'.'``
as ``path``). :param str path: path to be normalized
:return: normalized form of the given path (as a `str`) :raises IOError: if the path can't be resolved on the server
"""
path = self._adjust_cwd(path)
self._log(DEBUG, 'normalize(%r)' % path)
t, msg = self._request(CMD_REALPATH, path)
if t != CMD_NAME:
raise SFTPError('Expected name response')
count = msg.get_int()
if count != 1:
raise SFTPError('Realpath returned %d results' % count)
return msg.get_text() def chdir(self, path=None):
"""
Change the "current directory" of this SFTP session. Since SFTP
doesn't really have the concept of a current working directory, this is
emulated by Paramiko. Once you use this method to set a working
directory, all operations on this `.SFTPClient` object will be relative
to that path. You can pass in ``None`` to stop using a current working
directory. :param str path: new current working directory :raises IOError: if the requested path doesn't exist on the server .. versionadded:: 1.4
"""
if path is None:
self._cwd = None
return
if not stat.S_ISDIR(self.stat(path).st_mode):
raise SFTPError(errno.ENOTDIR, "%s: %s" % (os.strerror(errno.ENOTDIR), path))
self._cwd = b(self.normalize(path)) def getcwd(self):
"""
Return the "current working directory" for this SFTP session, as
emulated by Paramiko. If no directory has been set with `chdir`,
this method will return ``None``. .. versionadded:: 1.4
"""
# TODO: make class initialize with self._cwd set to self.normalize('.')
return self._cwd and u(self._cwd) def _transfer_with_callback(self, reader, writer, file_size, callback):
size = 0
while True:
data = reader.read(32768)
writer.write(data)
size += len(data)
if len(data) == 0:
break
if callback is not None:
callback(size, file_size)
return size def putfo(self, fl, remotepath, file_size=0, callback=None, confirm=True):
"""
Copy the contents of an open file object (``fl``) to the SFTP server as
``remotepath``. Any exception raised by operations will be passed
through. The SFTP operations use pipelining for speed. :param fl: opened file or file-like object to copy
:param str remotepath: the destination path on the SFTP server
:param int file_size:
optional size parameter passed to callback. If none is specified,
size defaults to 0
:param callable callback:
optional callback function (form: ``func(int, int)``) that accepts
the bytes transferred so far and the total bytes to be transferred
(since 1.7.4)
:param bool confirm:
whether to do a stat() on the file afterwards to confirm the file
size (since 1.7.7) :return:
an `.SFTPAttributes` object containing attributes about the given
file. .. versionadded:: 1.10
"""
with self.file(remotepath, 'wb') as fr:
fr.set_pipelined(True)
size = self._transfer_with_callback(
reader=fl, writer=fr, file_size=file_size, callback=callback
)
if confirm:
s = self.stat(remotepath)
if s.st_size != size:
raise IOError('size mismatch in put! %d != %d' % (s.st_size, size))
else:
s = SFTPAttributes()
return s def put(self, localpath, remotepath, callback=None, confirm=True):
"""
Copy a local file (``localpath``) to the SFTP server as ``remotepath``.
Any exception raised by operations will be passed through. This
method is primarily provided as a convenience. The SFTP operations use pipelining for speed. :param str localpath: the local file to copy
:param str remotepath: the destination path on the SFTP server. Note
that the filename should be included. Only specifying a directory
may result in an error.
:param callable callback:
optional callback function (form: ``func(int, int)``) that accepts
the bytes transferred so far and the total bytes to be transferred
:param bool confirm:
whether to do a stat() on the file afterwards to confirm the file
size :return: an `.SFTPAttributes` object containing attributes about the given file .. versionadded:: 1.4
.. versionchanged:: 1.7.4
``callback`` and rich attribute return value added.
.. versionchanged:: 1.7.7
``confirm`` param added.
"""
file_size = os.stat(localpath).st_size
with open(localpath, 'rb') as fl:
return self.putfo(fl, remotepath, file_size, callback, confirm) def getfo(self, remotepath, fl, callback=None):
"""
Copy a remote file (``remotepath``) from the SFTP server and write to
an open file or file-like object, ``fl``. Any exception raised by
operations will be passed through. This method is primarily provided
as a convenience. :param object remotepath: opened file or file-like object to copy to
:param str fl:
the destination path on the local host or open file object
:param callable callback:
optional callback function (form: ``func(int, int)``) that accepts
the bytes transferred so far and the total bytes to be transferred
:return: the `number <int>` of bytes written to the opened file object .. versionadded:: 1.10
"""
file_size = self.stat(remotepath).st_size
with self.open(remotepath, 'rb') as fr:
fr.prefetch(file_size)
return self._transfer_with_callback(
reader=fr, writer=fl, file_size=file_size, callback=callback
) return size def get(self, remotepath, localpath, callback=None):
"""
Copy a remote file (``remotepath``) from the SFTP server to the local
host as ``localpath``. Any exception raised by operations will be
passed through. This method is primarily provided as a convenience. :param str remotepath: the remote file to copy
:param str localpath: the destination path on the local host
:param callable callback:
optional callback function (form: ``func(int, int)``) that accepts
the bytes transferred so far and the total bytes to be transferred .. versionadded:: 1.4
.. versionchanged:: 1.7.4
Added the ``callback`` param
"""
with open(localpath, 'wb') as fl:
size = self.getfo(remotepath, fl, callback)
s = os.stat(localpath)
if s.st_size != size:
raise IOError('size mismatch in get! %d != %d' % (s.st_size, size)) ### internals... def _request(self, t, *arg):
num = self._async_request(type(None), t, *arg)
return self._read_response(num) def _async_request(self, fileobj, t, *arg):
# this method may be called from other threads (prefetch)
self._lock.acquire()
try:
msg = Message()
msg.add_int(self.request_number)
for item in arg:
if isinstance(item, long):
msg.add_int64(item)
elif isinstance(item, int):
msg.add_int(item)
elif isinstance(item, (string_types, bytes_types)):
msg.add_string(item)
elif isinstance(item, SFTPAttributes):
item._pack(msg)
else:
raise Exception('unknown type for %r type %r' % (item, type(item)))
num = self.request_number
self._expecting[num] = fileobj
self.request_number += 1
finally:
self._lock.release()
self._send_packet(t, msg)
return num def _read_response(self, waitfor=None):
while True:
try:
t, data = self._read_packet()
except EOFError as e:
raise SSHException('Server connection dropped: %s' % str(e))
msg = Message(data)
num = msg.get_int()
self._lock.acquire()
try:
if num not in self._expecting:
# might be response for a file that was closed before responses came back
self._log(DEBUG, 'Unexpected response #%d' % (num,))
if waitfor is None:
# just doing a single check
break
continue
fileobj = self._expecting[num]
del self._expecting[num]
finally:
self._lock.release()
if num == waitfor:
# synchronous
if t == CMD_STATUS:
self._convert_status(msg)
return t, msg
if fileobj is not type(None):
fileobj._async_response(t, msg, num)
if waitfor is None:
# just doing a single check
break
return None, None def _finish_responses(self, fileobj):
while fileobj in self._expecting.values():
self._read_response()
fileobj._check_exception() def _convert_status(self, msg):
"""
Raises EOFError or IOError on error status; otherwise does nothing.
"""
code = msg.get_int()
text = msg.get_text()
if code == SFTP_OK:
return
elif code == SFTP_EOF:
raise EOFError(text)
elif code == SFTP_NO_SUCH_FILE:
# clever idea from john a. meinel: map the error codes to errno
raise IOError(errno.ENOENT, text)
elif code == SFTP_PERMISSION_DENIED:
raise IOError(errno.EACCES, text)
else:
raise IOError(text) def _adjust_cwd(self, path):
"""
Return an adjusted path if we're emulating a "current working
directory" for the server.
"""
path = b(path)
if self._cwd is None:
return path
if len(path) and path[0:1] == b_slash:
# absolute path
return path
if self._cwd == b_slash:
return self._cwd + path
return self._cwd + b_slash + pathclass SFTPClient( )源码
- 基于用户名密码上传下载
- 基于公钥密钥上传下载
安装 paramiko
pip3 install paramiko
操作 paramiko
SSHClient
- SSHClient - 用于连接远程服务器并执行基本命令
import paramiko ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname='172.16.201.134', port=22, username='janice', password='janice123') stdin, stdout, stderr = ssh.exec_command('ls -la') results = stdout.read()
print(results.decode())
ssh.close()paramiko.SSHClient( )密码登入
import paramiko private_key = paramiko.RSAKey.from_private_key_file('/home/auto/.ssh/id_rsa') ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname='172.16.201.134', port=22, username='user', pkey=private_key) stdin, stdout, stderr = ssh.exec_command('ls -la') results = stdout.read()
print(results.decode())
ssh.close()paramiko.SSHClient( )公钥密钥连接
- 创建 Transport 对象来连接
import paramiko ip_port = ('172.16.201.134',22,)
transport = paramiko.Transport(ip_port)
transport.connect(username='user',password='user') ssh = paramiko.SSHClient()
ssh._transport = transport stdin, stdout, stderr = ssh.exec_command('df')
results = stdout.read() # 获取命令结果 print(results.decode())
transport.close()paramiko.Transport(ip_port)密码登入
import paramiko private_key = paramiko.RSAKey.from_private_key_file('/Users/jcchoiling/.ssh/vm.key') ip_port = ('172.16.201.134',22,)
transport = paramiko.Transport(ip_port)
transport.connect(username='user',pkey=private_key) ssh = paramiko.SSHClient()
ssh._transport = transport stdin, stdout, stderr = ssh.exec_command('df')
results = stdout.read() # 获取命令结果 print(results.decode())
transport.close()paramiko.Transport(ip_port)公钥密钥连接
SFPTClient
- SFTPClient - 用于连接远程服务器并执行上传下载
第一步：创建 transport 通道 e.g. paramiko.Transport(ip_port) 连接对象，负责上传文件到目标服务器端
第二步：transport 需要连接上服务器端，输入用户名和密码
第三步：创建 SFTPClient 对象然后把 transport作为参数传入 e.g. paramiko.SFTPClient.from_transport(transport)
第四步：可以调用 sftp.put/ sftp.get 方法来上传和下载文件
四、一）sftp.put(localfile, remotefile) 从本地上传到远端
四、二）sftp.get(remotefile, localfile) 从远端下载到本地
第五步：关闭 transport 通道import paramiko
ip_port = ('172.16.201.134',22,)
transport = paramiko.Transport(ip_port)
transport.connect(username='user',password='user') sftp = paramiko.SFTPClient.from_transport(transport)
sftp.put('/Users/jcchoiling/Desktop/movies.dat','/home/user/m1.dat')
sftp.get('/home/user/start.sh','/Users/jcchoiling/Desktop/start.sh') transport.close()paramiko.SFTPClient.from_transport(transport)密码登入
import paramiko private_key = paramiko.RSAKey.from_private_key_file('/Users/jcchoiling/.ssh/vm.key') ip_port = ('172.16.201.134',22,)
transport = paramiko.Transport(ip_port)
transport.connect(username='user',pkey=private_key) sftp = paramiko.SFTPClient.from_transport(transport)
sftp.put('/Users/jcchoiling/Desktop/movies.dat','/home/user/m1.dat')
sftp.get('/home/user/start.sh','/Users/jcchoiling/Desktop/start.sh') transport.close()paramiko.SFTPClient.from_transport(transport)公钥密钥连接
上下文操作应用
初探堡垒机
本周作业
作业：开发一个由数据库管理的主机管理系统,主机分组、分用户权限管理
- 所有的用户操作日志要保留在数据库中
- 每个用户登录堡垒机后，只需要选择具体要访问的设置，就连接上了，不需要再输入目标机器的访问密码
- 允许用户对不同的目标设备有不同的访问权限，例:
- 对10.0.2.34 有mysql 用户的权限
- 对192.168.3.22 有root用户的权限
- 对172.33.24.55 没任何权限
- 分组管理，即可以对设置进行分组，允许用户访问某组机器，但对组里的不同机器依然有不同的访问权限
考核题
试说明你写这个作业的思路：
- 首先看到管理主机组，这表明可能会有多于一台服务器，然后假设我是管理100台 Hadoop 服务器的管理员。
- 然后设计数据库的表结构：
- 有本地用户 User
- 管理组表 Groups
- 本地用户和组的关系表 User-Groups
- 服务器表 Hosts
- 远程用户表 RemoteUsers
- 管理组表、远程用户和服务器的关系表 Hosts-Groups-RemoteUsers
- 记录表 Audit Log
- 然后可以从功能方面思考：
- 查看用戶信息
- 创建群組
- 创建用戶
- 创建服務器
- 创建遠程用戶
- 刪除群組
- 刪除用戶
- 刪除服務器
- 新增用戶到指定群組
- 新增服務器到指定群組
- 初始化數據庫
- 刪除數據庫
- 遠程連接
- 最后是把功能的流程关连在一起，比如说，你会想像当管理员一打开程序，第一步是什么、下一步又会是什么，这样的思路去设计你程序的流程图。
- 用户登入 (登入需要认证，可以添加登入3次锁定帐号)
- 登入后看到功能列表选单
- 每一个功能用一个函数来表达
- 当程序遇上非如期的输入时的处理方法 (Error Handling)
- 优雅地退出程序 (Exit program)
[知识点：重点是如果用 Python 操作数据库，从数据库中读写数据。]
数据库表结构：
程序运行结果：
总结
第五阶段主要是学习了如何用 Python 来实现网络编程，第一部份是介绍了基本的网络协议，其中重点是 TCP/IP 协议，学习如何写服务器端和客户端的 socket，还数据可以通个 TCP/IP 来输送和接收数据；第二部份介绍了 Python中的线程、进程和协程。然后还学了线程锁与进程锁；学了自定义线程池。第三部份学了消息队列的概念，分别是 Python 内置的 Queue 功能和 RabbitMQ的功能。具体介绍了 RabbitMQ 的发布和订阅、主题模式和 RPC 通信。第四部份学了数据库的操作：分别是 MySQL 中的原生 SQL话句和如何用 Python 的 pymysql 来操作 MySQL 数据库。在了这个基础之后，便深入介绍 Python 中的 ORM，当中最具代表性的就是 SQLAlchemy 模块，学习如何用 Python 调用 SQLAlchemy 中的功能来操作 MySQL数据库，这是一个很有意思的单元。
參考资料
银角大王：1) MySQL 操作
金角大王：1) Python之路,Day10~11 - 那就做个堡垒机吧
2) python 之路，Day11 - sqlalchemy ORM
OReilly.Essential.SQLAlchemy