Fundamentals & Performance
For talks overview, click here.
Python 103: Memory Model & Best Practices
Language Internals, Intermediate
Description: There's a growing crowd of Python users who don't consider themselves beginners anymore. However some users at this stage discover odd behavior that's hard to explain. Why doesn't code behave like it should? Why doesn't "correct" code execute correctly? We'll focus on Python's object & memory model, addressing these issues directly. Let's empower attendees to not create these bugs to begin with!
Abstract: In "Python 101," you learned basic Python syntax. In "Python 102" (or equivalent in experience), you went further, exploring Python more deeply -- creating/using classes, decorators, files, other standard library or 3rd-party modules/packages -- and graduated from being purely a beginner. Because Python has been around the block for quite awhile now, there is a continuously growing number of "Python 103" programmers out there. Many are no longer new to the language, however, they have run into various issues, bugs, or odd behavior in their code that is difficult to explain. It's time to take a closer look. This is an interactive best practices talk, focusing on how Python objects, references, and the memory model work as well as thinking about performance. Knowing more about how the interpreter works under the covers, including the relationship between data objects and memory management, will make you a much more effective Python programmer, and the (main) goal with the knowledge imparted in this talk is to empower developers to not (inadvertently) create certain classes of bugs in their code to begin with! All you need to bring is the desire to learn more about the interpreter to take your Python skills to the next level.
Bio: Wesley J Chun is the author of the bestselling Core Python titles and the Python Fundamentals Live Lessons companion video. He is coauthor of Python Web Development with Django (withdjango.com), and has written for Linux Journal, CNET, and InformIT. Wesley is an architect and Developer Advocate at Google.
"Good Enough" IS Good Enough!
Description: Our culture's default assumption is that everybody should always be striving for perfection -- settling for anything less is seen as a regrettable compromise. This is wrong in most software development situations: focus instead on keeping the software simple, just "good enough", launch it early, and iteratively improve, enhance, and re-factor it. This is how software success is achieved!
Abstract: In 1989, Richard Gabriel caricatured two approaches to SW development: "worse is better" ("New Jersey approach") and "the right thing" ("MIT/Stanford approach"), reluctantly concluding NJ was more viable, for several reasons (speed of development, flexible designs, systems adaptable to a variety of uses [including changes in requirements], ease of gradual, incremental improvement, ...). And this debate hasn't died down since.
Debate rages, but reality has moved away from "right thing" ("Cathedral"-centralized "Big Design Up Front", focus on academia/large firms, unsuited to shifting real-world requirements), toward "NJ" ("Bazaar"-like, agile iterative enhancement, dynamic start-ups/independent developers, a world of always-shifting specs).
In this talk I support "the NJ approach", on both philosophical and pragmatical grounds, with examples from many areas. Winners of the "mind-share battles" focused on simplicity ("good enough"), not theoretical refinement/completeness: large ecosystems of developers, incremental improvement -- TCP/IP approach vs ISO/OSI, HTTP/HTML vs Xanadu, early Unix's simplistic (but OK) approach to interrupted system calls vs Multic's/ITS's perfectionism.
In Python, metaclasses often end up too complex (80% of their pluses can be had via class decorators, for 20% of the complexity); OTOH, incremental improvement worked just fine in sorting, generators, and guaranteed-finalization semantics.
The talk is not perfect, but I do think it's good enough.
Bio: Author of "Python in a Nutshell", co-author of "Python Cookbook", PSF Fellow, frequent speaker at Python conferences, prolific contributor to StackOverflow, and winner of the 2006 Frank Willison Memorial Award for contributions to Python, Alex currently leads "1:many tech support" for Google Cloud Platform. He's married to Anna Ravenscroft, his co-author in the "Cookbook" 2nd edition and "Nutshell" 3rd edition, also a PSF Fellow, and also a winner of the Frank Willison Memorial Award, in 2013.
Magic Considered Harmful?
Description: We talk about magic tongue-in-cheek and often with a negative connotation. This talk will look to clearly define magic and answer the question of "when is magic harmful?"
Abstract: This talk will look into some libraries we use (and love!) and the magic that powers them.
We will look at how things like namedtuple, Django models, and the Flask request object are implemented. We will look at how each of these libraries encapsulates complexity. At the end of the talk (hopefully) we will understand more about the internals of these libraries and have new tools to gauge if magic should be considered harmful for yourself.
The Tower of Abstraction
Description: Abstraction is a powerful servant, but a dangerous master. We code, design, think, debug ... on a tower of abstractions. Spolsky's Law tells us that "All abstractions leak". This talk explores why they leak, why that's often a problem, what to do about it; I also cover why sometimes abstractions SHOULD "leak", and how best to produce and consume abstraction layers.
The Python Deployment Albatross
Description: Python deployments can be notoriously tricky - a lot more trickier than they need to be. This talk will briefly outline the history of Python deployments, explore in depth the current landscape and will shed light on the variety of tools and techniques available to deploy Python applications, from the popular to the trendy/state-of-the-art to the esoteric and my experiences with them.
Abstract: To understand the current state of packaging, distribution and deployment of Python artifacts, it's important to understand the architecture and internals of commonly used tools such as distutils, setuptools, eggs, pip, PyPI, virtualenv, wheels and building compiled extensions.
In the recent few years, an interesting development from Twitter has been PEX - P(ython) EX(ecutable). PEX is famously being used at companies like Twitter, Square and LinkedIn to deploy *all* applications. This talk will chart our history of using PEX (along with the Pants build system) at imgix over the course of the last 2 years.
No talk on python packaging and distribution will be complete without mentioning the D word - yes, you guessed it right - Docker. This talk will explore the current state of Python deployment using Docker/containers as well as several anti patterns. The talk will highlight the challenges containerization calls for and why it might not be the right solution for many use cases. I'll be drawing on my experience running a Dockerized Tornado application in production.
Lastly, the talk will explore Nix - a powerful open source package manager for Linux and other Unix systems that makes package management reliable and reproducible. The talk will detail some of our experience with Nix so far as well as some of the issues we ran into, and whether Nix is a viable alternative to the existing tools in the ecosystem.
Bio: I've been working with Python for over 5 years now and am currently employed at imgix where I work on Python, Go and other misc backend stuff. I organize the San Francisco Python Twisted and Bay Area Lua Developers meetups.
Unspeakably Evil Hacks in Service of Marginally Improved Syntax: "Compile-Time" Python Programming
Language Internals, Intermediate
Description: One of Python's strengths as a dynamic language is its suite of powerful metaprogramming tools. What happens, however, when you want to move beyond the tools provided by "traditional" metaprogramming techniques? This talk will take the audience on a brief tour of projects and techniques that stretch the boundaries of what's possible in Python.
Abstract: In this talk, we provide an introduction to several lesser-known techniques for hacking extending the functionality of Python. Along the way, we consider the costs (in clarity, portability, or otherwise) of employing nonstandard tools to work around limitations of Python.
Topics may include:
- Runtime Bytecode Rewriting (https://github.com/llllllllll/codetransformer)
- Hooking the Lexer with Custom Encodings (https://github.com/dropbox/pyxl)
- Import Hooks (https://github.com/hylang/hy, http://cython.org/)
Bio: Scott Sanderson is an engineer at Quantopian, where he is responsible for the design of Quantopian's backtesting and research APIs. He is a core developer on the open source backtesting library Zipline, and he is a contributor to several projects in the PyData ecosystem, including IPython and the Jupyter Notebook. Scott graduated from Williams College in 2013 with bachelor's degrees in Mathematics and Philosophy.
Python Profiling and Performance: Elementary to Enterprise
Performant Python, Intermediate
Description: This talk provides an end-to-end introduction and overview of Python performance practices, from fundamentals to functional industry practices to the future of performant Python. If you've ever felt lost in or out of touch with the constant whirl of Python performance advancements, this practical talk will put it back into perspective.
Abstract: Performance is a complex topic. It means a lot of things to a lot of people. Python gives us a great starting point: strong primitives and the "good enough" philosophy. But is Python actually good enough for performance-critical applications?
This talk defines different kinds of performance, covers basic principles, and dives right into measurement. With those foundations laid, it outlines eight approaches to scaling Python, four of which are stack-agnostic and four of which are Python-specific. It outlines many examples from industry to promote a holistic view of performance as a practical process, not a large-scale benchmarking competition.
Bio: Mahmoud Hashemi is Lead Developer of Python Infrastructure at PayPal, where he focuses on distributed systems, API design, and application security. He presented O'Reilly's Enterprise Software with Python, as well as several guides to topics from DNS to software versioning to statistics. An avid Wikipedian, Mahmoud is half of Hatnote, creators of Listen to Wikipedia and other fine wiki-based software.
Pants, or How I Learned to Stop Worrying and Love Builds
Scalable Python, Intermediate
Description: For integrated services, it makes sense to keep several logical Python projects in a single repository -- a common library, a web front end and a back end service. For such repositories, Pants (build in Python for Python, Java, C++ and more) helps maintain dependencies and build (mostly) stand-alone executables which simplify deployment.
Abstract: Pants is a modern build system written in Python. It can build Python, Java, C++, Go and more. Twitter, Square and FourSquare use it internally, and contribute to it.
Bio: Moshe is a Twisted contributor, and has contributed to core Python. He loves infrastructure for building, monitoring and making services highly available.
Explore Git internals using Python | Let's write `git log` in Python
Description: Git is a powerful tool for source control. It's often misunderstood and abused. Under the surface Git is an elegant and simple data structure. When you don't understand that data structure, you don't really understand Git. It is flexible enough to give you all the rope that you need to hang yourself in Git hell. However, if you understand it, you are released from Git hell.
Abstract: In this talk, we start with a simple explanation of the Git data structure on disk. We discuss where the local Git repo is stored: `.git`. From there, we discuss the `config, `HEAD`, `refs/heads`, and `objects`.
We use Python to read those data structures and reconstruct a `git log` command for any arbitrary git repository. When finished, we should have our own working command that does the same thing as `git log` for any arbitrary repository, on any branch. We'll simply start at `HEAD` and work our way down the data structure.
Although it is not *useful* to have a Python version of Git, it is *fun*. Also, this exploration helps you understand the Git tool on a much deeper level. When you can program something, you can understand it. And, understanding Git helps you be a better developer and collaborator.
Bio: Glen Jarvis has been programming Python for over 8 years and has been programming in different languages for longer. He has been certified in Linux/Unix administration by UC-Berkeley. He gained the highest certification available for Informix DBAs. He is also certified in MongoDB as Developer and Administrator. He has worked for companies such as IBM, UC-Berkeley, Sprint and Silicon Valley Start-ups. He has worked in the fields of Databases, DataScience, Bioinformatics and Web Technologies.
One Pykid at a time
Description: If you are a Pythonista, an educator, STEM supporter, love free software and a parent then you should attend this talk. This talk brings home the importance of brining STEM and computing education to the K-12 school children early and in a timely manner.
Abstract: Python is a language that makes learning programming easy and can set the foundation for our children to go on and take STEM coursework or use their knowledge of computing in other subject areas when its time for graduate school. Our school system currently has a gap in their curriculum when it comes to computing and learning how to code. pykids is a voluntary organization that is aiming to fill that gap by providing easy to use learning resources. pykids is also encouraging classroom learning by creating a space for local meetups and volunteer classes that run through the curriculum.
The pykids set up today includes the following:
- A blog/website where students/instructors register and share ideas
- A jupyter server that allows running notebooks on the fly
- Downloadable Notebooks created for K3-High School students (WIP)
- Teaching material for K-3 students
All a student needs is a laptop and an internet connection to start learning Python!
Bio: Meenal Pant is a mom, long time programmer and yes a Pythonista!. She has worked in both the industry and academic /research institutes and therefore is keen to “build a bond” between technology and education. She is a poster presenter and speaker (education summit/lightening talks) in the past few PyCons - the most recent being PyCon2016. She is actively involved in the STEM education via her workplace and also personally in her kid’s schools.
Beautiful Documentation Oriented Programming
Description: Have you ever wonder how to write beautiful documentation with minimal effort? Did the tools get in your way in the process? This talk offers practical examples of leveraging simple text and docstrings to create stunning browsable documentation while making sure your code works as designed.
Abstract: Documentation is a fundamental organizational tool. Not only it help us to understand our programs, documentation can help us to develop and test our code iteratively.
Formats and tools like reStructuredText and Sphinx had made a positive lasting impact in our Python community as we can now easily write splendid documentation with little effort. In turn, the documentation can be auto-tested and taken straight from our source code avoiding redundancy.
This talk highlights the benefits of using simple text for writing programs and documentation, teaching the basics of reStructuredText, Sphinx, docstrings and doctests. We will be modeling the early stages of developing an application, following best practices, verifying program correctness and learning how to create beautiful documentation.
Bio: Daniel Mizyrycki has been programming in Python for over a decade in industry (GreenBusinessCA, Amazon, Docker) and educational (CCSF, RCSD) environments. Previously, he used assembly, C, perl, bash, founded the first Argentinean Linux User Group (1993) and consulted for early Argentinean ISPs. He loves Python's community being a PSF Contributing Member at SFPython, PyLadies, Baypiggies, PyCon and authoring sphinxserve and loadconfig. Today, he teaches Python to hundreds of Cisco engineers.
Self-Healing Systems: The Road to 99.99% Uptime
Scalable Python, Intermediate
Description: Stop firefighting and start fireproofing! There are many tools that make oncall easier and increase availability, but we'll be mostly focusing on a few principles and design patterns that help make your systems more robust.
Abstract: Feature velocity is typically a higher priority early in a software's lifecycle, but as the system matures there is an effort to start fireproofing the system. On the Yelp Transactions Platform team we've used a combination of circuit breakers, queues, and idempotent operations to minimize downtime and waking up in the middle of the night.
We'll take a look at how these design patterns help us in a distributed system, when they should be used, and common pitfalls associated.
Bio: William Ting is a longtime FOSS advocate with contributions in various projects (Pelican, autojump, pyramid_swagger, Rust, GNOME). He's currently an infrastructure engineer at Reddit, and previously on the Yelp Transaction Platform team.
Python tracing superpowers with systems tools
Performant Python, Intermediate
Description: Modern system tracers like SystemTap or Dtrace are incredibly powerful. If they're not part of your arsenal of techniques for analyzing Python code, you might be missing out. In this talk, we'll explore how these tools work, and how they can be used for dynamic, low-overhead analysis of unmodified Python programs.
Abstract: Maybe you want to profile your program, but it's running lots of C extension code and conventional profilers can't help you. Or maybe you're tracking down an emergent problem in a production system, but the logs are barren.
Advanced tracing toolkits like SystemTap can help you analyze your program in real time, without modifying or restarting it. But they can also seem dauntingly unfriendly, especially when applied to interpreted languages like Python.
Fear not! We'll talk about how kernel tracing actually works, what tools are available, and what we need to know about the Python interpreter's internals to use them effectively. We'll see how to do mixed-mode profiling, and how to trace specific events, like memory allocations or network calls. We'll discuss some of the pros and cons of these techniques, and how they can be applied to debugging systems in other languages too.
Bio: Eben is a software engineer based in San Francisco. He's used Python to do math research and build email infrastructure, among other things. He likes pie, and rock climbing.
Xonsh – put some Python in your Shell.
Description: Xonsh is a Python-ish, BASHwards-looking shell language and command prompt. The language is a superset of Python 3.4+ with additional support for the best parts of shells that you are used to, such as Bash, zsh, fish, and IPython. It works on all major systems including Linux, Mac OSX, and Windows. Xonsh is meant for the daily use of experts and novices alike.
Abstract: Programmers spend their time at a command line interface often sticking to
default shell. A lot of progress have been made for the friendliness,
usability, extensibility of shell. We thus introduce Xonsh which attempt to
bring the command line shell to the 21st century.
Xonsh is general purpose shell that combines Python and the best features of
Bash, zsh, IPython and fish. Written in Python and relying only the standard
library and PLY, the xonsh language is a strict superset of Python that
compiles to a Python AST. The shell can provides exciting features: rich
history, tab completion from bash and man pages, syntax highlighting,
auto-suggestion, foreign-function aliases and more!
Wether you are a novice who is looking to use use the command line, or an
Python expert Xonsh is made for you.
Because xonsh is Python, it automatically has all the available python
ecosystem at your fingertip. Xonsh makes meshing and intertwining python code
with command-line interfaces as seamless as possible. Have you ever wanted to
use regular expressions to glob files? No problem! Ever wanted to curl a remote
resource right into `json.loads()`? Now you can. Do you not want to leave the
command line to use pandas, NLTK or add two numbers together? No big deal.
The xonsh homepage is at https://xon.sh
Bio: I am a PostDoc at UC Berkeley Institute for Data science, and have been a core Developer of IPython and Jupyter for a couple of years. With a background in Physics I spend most of my time developing tools for the scientific community and for education as well as promoting Python 3.
A Guide to Bad Programming
Description: In a sea of talks and information about how to improve your coding skills, this talk will make a case for bad code in your everyday life. In this talk you'll learn how and why you should write bad code.
Abstract: Inspired by "the queen of sh*tty robots" and a talk I had recently with a friend about how often our code "optimizations" and best practices don't matter, this talk will point out some of the obsessions we have as professional programmers that don't matter and can even be harmful to the progress of a product. The talk will show how identifying as a "bad programmer" can improve your skills in the long run and help you become a better programmer. Lastly, the talk will showcase some "bad practices" that can be fine or even good when used appropriately.
Bio: I'm a web developer with a background in aerospace engineering. I'm obsessed with Web technology and created an award winning Chrome application called Neutron Drive (https://super.neutrondrive.com/). I also run the PyWeb Meetup in Houston TX and am a chair person for the PyTexas annual conference. In addition to being a Web and aerospace geek, I'm a father of three and can cook a pretty mean pizza from scratch. I hold a BS in Aerospace Engineering from Embry-Riddle Aeronautical University.
Turnkey Distributed Tracing: OpenTracing and Python
Performant Python, Intermediate
Description: This talk underlines the importance of distributed tracing in microservices, reveals the need for standardization of tracing instrumentation, and explains how the OpenTracing project addresses that need. We will showcase OpenTracing’s set of consistent, expressive, explicitly vendor-neutral tracing APIs with working examples using python gRPC, Flask, and possibly other frameworks if time allows.
Abstract: Microservice architectures offer many benefits but are notoriously difficult to observe or debug, especially as transactions inevitably cross process boundaries. I focused on this problem at Google and built Dapper, Google’s production distributed tracing system. I learned from Dapper’s design eccentricities that—in order to facilitate ubiquitous, transparent distributed tracing in OSS-heavy applications—as an industry we must find a way to make open-source software components interoperate from a tracing standpoint.
This need for standardized tracing instrumentation across open source brings us to OpenTracing. By offering consistent, expressive, vendor-neutral, open-source APIs for popular platforms, OpenTracing provides a semantic and syntactic standard for distributed tracing instrumentation, allowing developers to add (or switch) tracing implementations with an O(1) configuration change.
I will explain OpenTracing’s data model and the motivations behind it, describe how it generalizes across tracing systems and instrumentation languages (with a focus on Python), and show demos of how it integrates with frameworks (like Flask and gRPC) and applications. We will then cover the “hello world” of tracing: creating and linking traces that traverse process boundaries. Finally, I’ll share what’s in store for the rest of 2016 and 2017 and talk about ways to get involved.
Bio: Ben is a cofounder at LightStep, an early-stage company focused on reliability and performance management for modern distributed systems. He is a globally recognized expert in monitoring and tracing for highly concurrent, large-scale distributed systems. Previously, Ben was at Google where he built and led the engineering teams for Dapper — Google’s distributed systems tracing infrastructure — and Monarch, Google’s fleet-wide timeseries collection, storage, analysis, and alerting system.