Python全栈(四)高级编程技巧之2.类与对象深度问题与解决技巧 - Go语言中文社区

Python全栈(四)高级编程技巧之2.类与对象深度问题与解决技巧


一、派生内置不可变类型并修改其实例化行为

引入:
我们想自定义一种新类型的元组,对于传入的可迭代对象,我们只保留其中int类型且值大于0的元素,例如:
IntTuple([2,-2,'jr',['x','y'],4]) => (2,4)
如何继承内置tuple 实现IntTuple?
自定义IntTuple:

class IntTuple(tuple):
    def __init__(self,iterable):
        for i in iterable:
            if isinstance(i,int) and i > 0:
                super().__init__(i)

int_t = IntTuple([2,-2,'cl',['x','y'],4])
print(int_t)

打印

Traceback (most recent call last):
  File "xxx/demo.py", line 7, in <module>
    int_t = IntTuple([2,-2,'cl',['x','y'],4])
  File "xxx/demo.py", line 5, in __init__
    super().__init__(i)
TypeError: object.__init__() takes exactly one argument (the instance to initialize)

问题:self对象到底是谁创建的:

class A:
    def __new__(cls, *args, **kwargs):
        print('A.__new__',cls,args)
        return object.__new__(cls)

    def __init__(self,*args):
        print('A.__init__')

a = A(1,2)

打印

A.__new__ <class '__main__.A'> (1, 2)
A.__init__

易知,真正创建对象的是__new__方法,self对象是通过__new__创建的
上述示例默认继承object,也可以继承自其他类,__new__方法返回的是super().new(cls)。

class B:
    pass

class A(B):
    def __new__(cls, *args, **kwargs):
        print('A.__new__',cls,args)
        return super().__new__(cls)

    def __init__(self,*args):
        print('A.__init__')

a = A(1,2)

运行结果与前者相同,但最好采用第一种方式。
A的实例化可以转化为另两行等价的代码:

class A:
    def __new__(cls, *args, **kwargs):
        print('A.__new__',cls,args)
        return object.__new__(cls)

    def __init__(self,*args):
        print('A.__init__')

# a = A(1,2)相当于下面两行代码
a = A.__new__(A,1,2)
A.__init__(a,1,2)

创建列表转化等价代码:

l = list.__new__(list,'abc')
print(l)
list.__init__(l,'abc')
print(l)

打印

[]
['a', 'b', 'c']

但是元组不同:

t = tuple.__new__(tuple,'abc')
print(t)
tuple.__init__(t,'abc')
print(t)

打印

('a', 'b', 'c')
('a', 'b', 'c')

在第一步调用__new__方法时就已经实例化生成了元组。
这也解释了在最开始自定义IntTuple时__init__(self,iterable)会报错:
在__init__(self,iterable)之前调用__new__方法时元组已经被创建好了,并且元组是不能被修改的,所以会报错,不能实现预期效果。
对代码的修改:

class IntTuple(tuple):
    def __new__(cls,iterable):
        #生成器
        r = (i for i in iterable if isinstance(i,int) and i > 0)
        return super().__new__(cls,r)

int_t = IntTuple([2,-2,'cl',['x','y'],4])
print(int_t)

打印

(2, 4)

实现了预期功能。

二、创建大量实例节省内存

问题提出:
在游戏中,定义了玩家类player,每有一个在线玩家,在服务器内则有一个player的实例,当在线人数很多时,将产生大量实例(百万级),如何降低这些大量实例的内存开销?
解决方案:
定义类的__slots__属性,声明实例有哪些属性(关闭动态绑定)。

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

p1 = Player1('0001','Tom')
p2 = Player2('0002','Tom')
print(len(dir(p1)))
print(len(dir(p2)))
print(dir(p1))
print(dir(p2))
print(set(dir(p1)) - set(dir(p2)))

打印

30
29
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'level', 'name', 'status', 'uid']
['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', 'level', 'name', 'status', 'uid']
{'__dict__', '__weakref__'}

显然,p1和p2的所有属性相比较,p2比p1少了属性,其中__weakref__是弱引用,__dict__是动态绑定属性,主要的内存浪费就在于此。

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

p1 = Player1('0001','Tom')
p1.x = 6
p1.__dict__['y'] = 7 #p1.__dict__本身就是字典
print(p1.__dict__)

打印

{'uid': '0001', 'name': 'Tom', 'status': 0, 'level': 1, 'x': 6, 'y': 7}

可知,增加了属性x和y,此即动态绑定,即可以随意向对象添加属性。
Python可以通过这种动态绑定添加属性,但是这种方式是及其浪费内存的,添加__slots__后可消除动态绑定、解决这个问题。

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

p1 = Player1('0001','Tom')
p2 = Player2('0002','Tom')
p1.x = 6
p2.y = 7
print(p1.__dict__)
print(p2.__dict__)

打印

Traceback (most recent call last):
  File "xxx/demo.py", line 73, in <module>
    p2.y = 7
AttributeError: 'Player2' object has no attribute 'y'

即不允许再向Player2的实例任意添加属性。
导入sys模块查看具体内存:

import sys

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

p1 = Player1('0001','Tom')
p1.x = 6
print(p1.__dict__)
print(sys.getsizeof(p1.__dict__))
print(sys.getsizeof(p1.uid))
print(sys.getsizeof(p1.name))
print(sys.getsizeof(p1.status))
print(sys.getsizeof(p1.level))
print(sys.getsizeof(p1.x))

打印

112
53
52
24
28
28

不允许对实例动态修改__slots__:

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

p1 = Player1('0001','Tom')
p2 = Player2('0002','Tom')
p1.x = 6
p1.__dict__['y'] = 7
p2.__slots__ = ('uid','name','status','level','y')
print(p1.__dict__)
print(p2.__dict__)

打印

Traceback (most recent call last):
  File "xxx/demo.py", line 74, in <module>
    p2.__slots__ = ('uid','name','status','level','y')
AttributeError: 'Player2' object attribute '__slots__' is read-only

报错,__slots__是只读的。
导入库tracemalloc跟踪内存的使用:

import tracemalloc

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

tracemalloc.start()
p1 = [Player1('0001','Tom',2,3) for _ in range(100000)]
p2 = [Player1('0001','Tom',2,3) for _ in range(100000)]
end = tracemalloc.take_snapshot()
top = end.statistics('lineno')

for stat in top[:10]:
    print(stat)

打印

xxx/demo.py:59: size=21.4 MiB, count=399992, average=56 B
xxx/demo.py:74: size=6274 KiB, count=100003, average=64 B
xxx/demo.py:73: size=6274 KiB, count=100001, average=64 B
xxx/demo.py:75: size=432 B, count=1, average=432 B
xxxPythonPython37libtracemalloc.py:532: size=64 B, count=1, average=64 B

end.statistics()的参数为’lineno’时是对占内存的代码逐行分析:
59行和73行是对p1进行分析,占内存为21.4 MiB(__dict__属性内存)+6274 KiB=27.5MiB;
74行是对p2进行分析:6274 KiB=6.1MiB。
end.statistics()的参数为’filename’时是对整个文件内存进行分析:
此时需要对p1和p2分别分析:
对p1:

import tracemalloc

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

tracemalloc.start()
p1 = [Player1('0001','Tom',2,3) for _ in range(100000)]
# p2 = [Player1('0001','Tom',2,3) for _ in range(100000)]
end = tracemalloc.take_snapshot()
# top = end.statistics('lineno')
top = end.statistics('filename')

for stat in top[:10]:
    print(stat)

打印

xxx/demo.py:0: size=16.8 MiB, count=299994, average=59 B
xxxPythonPython37libtracemalloc.py:0: size=64 B, count=1, average=64 B

对p2:

import tracemalloc

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

tracemalloc.start()
# p1 = [Player1('0001','Tom',2,3) for _ in range(100000)]
p2 = [Player1('0001'
                        
                        
版权声明:本文来源CSDN,感谢博主原创文章,遵循 CC 4.0 by-sa 版权协议,转载请附上原文出处链接和本声明。
原文链接:https://blog.csdn.net/CUFEECR/article/details/104117162
站方申明:本站部分内容来自社区用户分享,若涉及侵权,请联系站方删除。

0 条评论

请先 登录 后评论

官方社群

GO教程

猜你喜欢