Python自学：使用多进程处理 multiprocessing-Toy模板网

这篇具有很好参考价值的文章主要介绍了Python自学：使用多进程处理 multiprocessing。希望对大家有所帮助。如果存在错误或未考虑完全的地方，请大家不吝赐教，您也可以点击"举报违法"按钮提交疑问。

1. 使用多进程执行函数

以下代码没有使用多进程。

import time

start = time.perf_counter()

def do_something():
    print('Sleeping 1 second...')
    time.sleep(1)
    print('Done Sleep...')

do_something()
do_something()

finish = time.perf_counter()

print(f'Finished in {round(finish-start, 2)} second(s)')

输出为：
Sleeping 1 second…
Done Sleep…
Sleeping 1 second…
Done Sleep…
Finished in 2.03 second(s)

以下代码使用了多进程。

import time
import multiprocessing


def do_something():
    print('Sleeping 1 second...')
    time.sleep(1)
    print('Done Sleep...')


if __name__ == '__main__':

    start = time.perf_counter()

    p1 = multiprocessing.Process(target=do_something)
    p2 = multiprocessing.Process(target=do_something)

    p1.start()
    p2.start()

    p1.join()
    p2.join()

    finish = time.perf_counter()

    print(f'Finished in {round(finish-start, 2)} second(s)')

输出为：
Sleeping 1 second…
Sleeping 1 second…
Done Sleep…
Done Sleep…
Finished in 1.07 second(s)

2. 使用loop创建多个进程，并在函数中传入参数。

import time
import multiprocessing


def do_something(seconds):
    print(f'Sleeping {seconds} second(s)...')
    time.sleep(seconds)
    print('Done Sleep...')


if __name__ == '__main__':

    start = time.perf_counter()

    processes = []

    for _ in range(10):
        p = multiprocessing.Process(target=do_something, args=[1.5])
        p.start()
        processes.append(p)

    for process in processes:
        process.join()

    finish = time.perf_counter()

    print(f'Finished in {round(finish-start, 2)} second(s)')

输出为：
Sleeping 1.5 second(s)…
Sleeping 1.5 second(s)…
Sleeping 1.5 second(s)…
Sleeping 1.5 second(s)…
Sleeping 1.5 second(s)…
Sleeping 1.5 second(s)…
Sleeping 1.5 second(s)…
Sleeping 1.5 second(s)…
Sleeping 1.5 second(s)…
Sleeping 1.5 second(s)…
Done Sleep…
Done Sleep…
Done Sleep…
Done Sleep…
Done Sleep…
Done Sleep…
Done Sleep…
Done Sleep…
Done Sleep…
Done Sleep…
Finished in 1.62 second(s)

3. 使用进程池实现多进程

import time
import concurrent.futures


def do_something(seconds):
    print(f'Sleeping {seconds} second(s)...')
    time.sleep(seconds)
    return f'Done Sleep...{seconds}'


if __name__ == '__main__':

    start = time.perf_counter()

    with concurrent.futures.ProcessPoolExecutor() as executor:
        secs = [5, 4, 3, 2, 1]
        results = executor.map(do_something, secs)

        for result in results:
            print(result)

    finish = time.perf_counter()

    print(f'Finished in {round(finish-start, 2)} second(s)')

输出为：
Sleeping 5 second(s)…
Sleeping 4 second(s)…
Sleeping 3 second(s)…
Sleeping 2 second(s)…
Sleeping 1 second(s)…
Done Sleep…5
Done Sleep…4
Done Sleep…3
Done Sleep…2
Done Sleep…1
Finished in 5.14 second(s)

4. 使用多进程处理图片

以下代码展示了没有使用多进程处理图片

import time
from PIL import Image, ImageFilter

img_names = [
    'photo-1516117172878-fd2c41f4a759.jpg',
    'photo-1532009324734-20a7a5813719.jpg',
    'photo-1524429656589-6633a470097c.jpg',
    'photo-1530224264768-7ff8c1789d79.jpg',
    'photo-1564135624576-c5c88640f235.jpg',
    'photo-1541698444083-023c97d3f4b6.jpg',
    'photo-1522364723953-452d3431c267.jpg',
    'photo-1493976040374-85c8e12f0c0e.jpg',
    'photo-1504198453319-5ce911bafcde.jpg',
    'photo-1530122037265-a5f1f91d3b99.jpg',
    'photo-1516972810927-80185027ca84.jpg',
    'photo-1550439062-609e1531270e.jpg',
    'photo-1549692520-acc6669e2f0c.jpg'
]

t1 = time.perf_counter()

size = (1200, 1200)

for img_name in img_names:
    img = Image.open(img_name)

    img = img.filter(ImageFilter.GaussianBlur(15))

    img.thumbnail(size)

    img.save(f'processed/{img_name}')
    print(f'{img_name} was processed...')

t2 = time.perf_counter()

print(f'Finished in {t2-t1} seconds')

输出为：
photo-1516117172878-fd2c41f4a759.jpg was processed…
photo-1532009324734-20a7a5813719.jpg was processed…
photo-1524429656589-6633a470097c.jpg was processed…
photo-1530224264768-7ff8c1789d79.jpg was processed…
photo-1564135624576-c5c88640f235.jpg was processed…
photo-1541698444083-023c97d3f4b6.jpg was processed…
photo-1522364723953-452d3431c267.jpg was processed…
photo-1493976040374-85c8e12f0c0e.jpg was processed…
photo-1504198453319-5ce911bafcde.jpg was processed…
photo-1530122037265-a5f1f91d3b99.jpg was processed…
photo-1516972810927-80185027ca84.jpg was processed…
photo-1550439062-609e1531270e.jpg was processed…
photo-1549692520-acc6669e2f0c.jpg was processed…
Finished in 13.196055100299418 seconds

使用多进程的方式处理图片

import time
import concurrent.futures
from PIL import Image, ImageFilter

img_names = [
    'photo-1516117172878-fd2c41f4a759.jpg',
    'photo-1532009324734-20a7a5813719.jpg',
    'photo-1524429656589-6633a470097c.jpg',
    'photo-1530224264768-7ff8c1789d79.jpg',
    'photo-1564135624576-c5c88640f235.jpg',
    'photo-1541698444083-023c97d3f4b6.jpg',
    'photo-1522364723953-452d3431c267.jpg',
    'photo-1493976040374-85c8e12f0c0e.jpg',
    'photo-1504198453319-5ce911bafcde.jpg',
    'photo-1530122037265-a5f1f91d3b99.jpg',
    'photo-1516972810927-80185027ca84.jpg',
    'photo-1550439062-609e1531270e.jpg',
    'photo-1549692520-acc6669e2f0c.jpg'
]


def process_image(img_name):
    
    img = Image.open(img_name)

    img = img.filter(ImageFilter.GaussianBlur(15))

    img.thumbnail((1200, 1200))

    img.save(f'processed/{img_name}')
    print(f'{img_name} was processed...')

if __name__ == '__main__':
    t1 = time.perf_counter()

    with concurrent.futures.ProcessPoolExecutor() as executor:
        executor.map(process_image, img_names)

    t2 = time.perf_counter()

    print(f'Finished in {t2-t1} seconds')

输出为：
photo-1516117172878-fd2c41f4a759.jpg was processed…
photo-1516972810927-80185027ca84.jpg was processed…
photo-1524429656589-6633a470097c.jpg was processed…
photo-1522364723953-452d3431c267.jpg was processed…
photo-1532009324734-20a7a5813719.jpg was processed…
photo-1530122037265-a5f1f91d3b99.jpg was processed…
photo-1530224264768-7ff8c1789d79.jpg was processed…
photo-1564135624576-c5c88640f235.jpg was processed…
photo-1550439062-609e1531270e.jpg was processed…
photo-1541698444083-023c97d3f4b6.jpg was processed…
photo-1549692520-acc6669e2f0c.jpg was processed…
photo-1504198453319-5ce911bafcde.jpg was processed…
photo-1493976040374-85c8e12f0c0e.jpg was processed…
Finished in 2.651644399855286 seconds文章来源地址https://www.toymoban.com/news/detail-703207.html