Software Development
Advanced Python: Concurrency and Performance
For engineers whose Python services and pipelines have hit a wall. The course starts with measurement — cProfile, py-spy, tracemalloc — then works through the GIL, threads, process pools, and asyncio, before covering NumPy vectorization, caching, and native acceleration for the paths that remain too slow.
Overview
Practical learning for workplace transfer.
Performance work usually goes wrong in the same way: a team rewrites the loop that looks slow, ships it, and the endpoint is still timing out because the real cost was a serialization step or a connection pool of size five. This course insists on evidence first. Participants profile a live workload, classify it as CPU-bound, I/O-bound, or memory-bound, and only then choose between threads, processes, and asyncio. Concurrency correctness gets equal weight: race conditions, deadlocks, backpressure, cancellation, and the intermittent failures that are impossible to reproduce without a deliberate technique.
Prerequisites
Two or more years of Python development, including classes, generators, and experience running Python in production or in scheduled pipelines.
Objectives
- Profile CPU, memory, and wall-clock behavior to locate a bottleneck with evidence.
- Explain what the global interpreter lock blocks and choose threads or processes accordingly.
- Build asyncio services that sustain thousands of concurrent I/O operations safely.
- Diagnose race conditions, deadlocks, and cancellation bugs in concurrent code.
- Replace interpreted hot loops with NumPy vectorization, caching, or native extensions.
- Size workers, pools, and memory so throughput improves without destabilizing production.
Target audience
- Senior backend developers maintaining Python services under load
- Data engineers running batch and streaming pipelines that miss their windows
- Platform and SRE engineers investigating latency and resource exhaustion
- Machine learning engineers optimizing inference and feature pipelines
- Quantitative and scientific developers with numerically heavy workloads
- Architects deciding where Python should scale and where it should not
Program outline
A clear structure for the learning journey.
Outline points are grouped in one designed block instead of being treated as separate module cards.
01Measuring Before Optimizing4 topics
- Profiling with cProfile, py-spy, and line-level timing on a live workload
- Memory profiling with tracemalloc and finding unintended object retention
- Benchmarking honestly with timeit and stable measurement conditions
- Separating CPU-bound, I/O-bound, and memory-bound bottlenecks
02The GIL, Threads, and Processes4 topics
- What the global interpreter lock actually serializes and what it leaves free
- threading for I/O concurrency and the point where it stops helping
- multiprocessing, process pools, and the hidden cost of pickling data
- concurrent.futures as one interface over both execution models
03Asynchronous Python with asyncio4 topics
- The event loop, coroutines, tasks, and what await actually suspends
- Driving thousands of concurrent HTTP calls with httpx or aiohttp
- Timeouts, cancellation semantics, and gather compared with TaskGroup
- Calling blocking libraries from async code without stalling the loop
04Correctness Under Concurrency4 topics
- Race conditions, locks, semaphores, and the classic deadlock patterns
- Queues, worker pools, and applying backpressure to fast producers
- Idempotency and safe retry semantics inside concurrent pipelines
- Reproducing and debugging intermittent concurrency failures on purpose
05Making Python Code Fast4 topics
- Algorithmic complexity and data-structure choices that dominate runtime
- Vectorizing numeric work with NumPy instead of interpreted loops
- Caching with functools.lru_cache and with an external cache layer
- Native acceleration compared: C extensions, Cython, and Numba
06Scaling and Behavior in Production4 topics
- Sizing workers, connection pools, and memory budget per process
- Streaming and chunking to keep memory flat as volume grows
- Load testing a Python service and interpreting latency percentiles
- Optimizing a real workload from first measurement to verified gain
Materials provided
- Course workbook, annotated code samples, and reference notes
- Hands-on lab environment and starter repositories
- Exercises, checklists, and reusable code templates
- 4D Certificate of Completion
- Post-course technical guidance
Training Options
Programs can be delivered in-house, online, or in a blended format depending on your team's schedule, location, and learning objectives. When an external certificate or exam is included, certification rules and fees remain under the relevant awarding body's policies, while 4D provides the training and preparation support.
Why choose 4D
4D profiles one of your own slow endpoints or pipeline stages during the course, so participants see the difference between the bottleneck they assumed and the one the profiler shows. The optimization work is carried through to a measured before-and-after figure that engineering managers can put in front of a budget conversation about infrastructure spend.
Related courses
Python Programming Fundamentals
A hands-on introduction to Python for engineers and analysts who need working code rather than tutorial fragments. Participants build fluency with data types, collections, functions, files, and error handling, then assemble a small program that reads real data and produces a usable report.
View courseIntermediate Python and Clean Code Practices
Designed for developers whose Python works but is difficult to change. The course covers dataclasses, generators, decorators, context managers, and type hints, then applies the refactoring moves that turn a nine-hundred-line module into small tested units guarded by black, ruff, and mypy.
View coursePython Automation and Scripting for Work Tasks
Aimed at teams losing hours every week to repetitive file, spreadsheet, and reporting work. Participants automate those tasks with Python: batch file handling, Excel and CSV processing, REST API calls, email alerts, and scheduled jobs that run unattended and report clearly when something fails.
View course