Python Tips and Tricks (WIP)¶
Create virtualenv
Without Internet¶
To create the virtualenv without it choking, you will first need to pre-download the wheels of at least pip, setuptools, and wheel packages, then you can use the following:
virtualenv --python=<python-to-use> --no-download --extra-search-dir=/path/to/preinstall/wheels ENV_NAME
Now to install local packages you can:
pip install --no-index --find-links=/path/to/pre-downloaded/packages <package-to-install>
# --or--
pip install --no-index --find-links=/path/to/pre-downloaded/packages -r requirements.txt
I’ve used these myself for a Raspberry Pi that didn’t have an internet connection yet.
defaultdict
with functools.partial
for arguments¶
You can create a defaultdict
instance which will create objects with a
specific structure (OrderedDict
is a great use-case) by using
functools.partial
to initialize the object since defaultdict
will
construct it without any arguments:
def get_defaultdict_schema():
return OrderedDict([
('stuff', None),
('things', defaultdict(functools.partial(OrderedDict, [
('one_fish', None),
('two_fish', None),
('red_fish', None),
('blue_fish', False),
('enabled', True),
]))),
('enabled', True),
])
def get_nested_defaultdict_schema():
return defaultdict(functools.partial(OrderedDict, [
('stuff', None),
('things', defaultdict(functools.partial(OrderedDict, [
('one_fish', None),
('two_fish', None),
('red_fish', None),
('blue_fish', False),
('enabled', True),
]))),
('enabled', True),
]))
Code Profiling¶
You can use the cProfile
module in Python to perform code profiling, then
print out the information after your code has run:
import cProfile
import pstats
profiler = cProfile.Profile()
profiler.enable()
log.warning('PROFILING IS ENABLED!')
# DO STUFF YOU WANT TO PROFILE HERE
profiler.disable()
sortby = 'tottime'
stats_stream = io.StringIO()
stats = pstats.Stats(
profiler, stream=stats_stream
).sort_stats(sortby)
stats.print_stats()
stats_lines = stats_stream.getvalue().split('\n')
log.info('\n'.join(itertools.chain.from_iterable([
['\n'], [stats_lines[0]], stats_lines[4:25]
])))