Python

Version Update

Newer versions of python can focus heavily on performance improvements, some more than others. But be aware regressions for stability or security reasons may always be the case.

As a general rule, Python 3.9 and Python 3.11 both had a strong focus on performance improvements. Python 3.12 is looking promising but has yet to have widespread testing, adoption and deployment. A simple comparison is available at speed.python.org.

Djangobench is the source of synthetic benchmarks and a useful tool for running comparisons. Below are some examples to inform your investigations.

Keep in mind while a 1.2x faster result is significant, it’s only one step of the process, Celery, SQL, Redis, and many other factors will influence the endresult, and this python speed improvement will not translate 1:1 into real world performance.

Django 4.0.10

• Djangobench 0.10.0

• Django 4.0.10

• Python 3.8.18 vs Python 3.11.5

joel@METABOX:~/djangobench/django$ djangobench --vcs=none --control=. --experiment=. --control-python=/home/joel/djangobench/py38/bin/python --experiment-python=/home/joel/djangobench/py311/bin/python -r /home/joel/djangobench/results -t 500
Running all benchmarks
Recording data to '/home/joel/djangobench/results'
Control: Django 4.0.10 (in .)
Experiment: Django 4.0.10 (in .)

Running 'multi_value_dict' benchmark ...
Min: 0.000014 -> 0.000013: 1.0304x faster
Avg: 0.000183 -> 0.000133: 1.3698x faster
Significant (t=9.325958)
Stddev: 0.00010 -> 0.00007: 1.3791x smaller (N = 500)

Running 'query_values' benchmark ...
Min: 0.000079 -> 0.000070: 1.1308x faster
Avg: 0.000084 -> 0.000074: 1.1267x faster
Significant (t=19.174361)
Stddev: 0.00001 -> 0.00001: 1.0255x larger (N = 500)

Running 'query_delete' benchmark ...
Min: 0.000082 -> 0.000074: 1.1145x faster
Avg: 0.000086 -> 0.000078: 1.0987x faster
Significant (t=17.504085)
Stddev: 0.00001 -> 0.00001: 1.1888x smaller (N = 500)

Running 'query_select_related' benchmark ...
Min: 0.016771 -> 0.013520: 1.2405x faster
Avg: 0.017897 -> 0.014149: 1.2649x faster
Significant (t=40.942990)
Stddev: 0.00190 -> 0.00077: 2.4535x smaller (N = 500)

Running 'query_aggregate' benchmark ...
Min: 0.000092 -> 0.000083: 1.1105x faster
Avg: 0.000100 -> 0.000090: 1.1107x faster
Significant (t=9.967204)
Stddev: 0.00002 -> 0.00001: 1.5003x smaller (N = 500)

Running 'query_raw_deferred' benchmark ...
Min: 0.004157 -> 0.003563: 1.1666x faster
Avg: 0.004626 -> 0.003809: 1.2143x faster
Significant (t=12.325104)
Stddev: 0.00121 -> 0.00086: 1.4047x smaller (N = 500)

Running 'query_get_or_create' benchmark ...
Min: 0.000412 -> 0.000362: 1.1385x faster
Avg: 0.000458 -> 0.000407: 1.1259x faster
Significant (t=14.169322)
Stddev: 0.00006 -> 0.00005: 1.1306x smaller (N = 500)

Running 'query_values_list' benchmark ...
Min: 0.000080 -> 0.000071: 1.1231x faster
Avg: 0.000089 -> 0.000076: 1.1706x faster
Significant (t=18.298942)
Stddev: 0.00001 -> 0.00001: 1.9398x smaller (N = 500)

Running 'url_resolve_flat_i18n_off' benchmark ...
Min: 0.055764 -> 0.045370: 1.2291x faster
Avg: 0.057670 -> 0.047020: 1.2265x faster
Significant (t=111.187780)
Stddev: 0.00206 -> 0.00059: 3.4618x smaller (N = 500)

Running 'qs_filter_chaining' benchmark ...
Min: 0.000236 -> 0.000196: 1.2034x faster
Avg: 0.000248 -> 0.000206: 1.2041x faster
Significant (t=44.893544)
Stddev: 0.00002 -> 0.00001: 1.0833x smaller (N = 500)

Running 'template_render' benchmark ...
Min: 0.000933 -> 0.000712: 1.3110x faster
Avg: 0.001003 -> 0.000777: 1.2909x faster
Significant (t=8.379095)
Stddev: 0.00043 -> 0.00042: 1.0287x smaller (N = 500)

Running 'query_get' benchmark ...
Min: 0.000259 -> 0.000230: 1.1259x faster
Avg: 0.000282 -> 0.000238: 1.1829x faster
Significant (t=42.267305)
Stddev: 0.00002 -> 0.00001: 1.7842x smaller (N = 500)

Running 'query_none' benchmark ...
Min: 0.000053 -> 0.000045: 1.1830x faster
Avg: 0.000056 -> 0.000049: 1.1449x faster
Significant (t=16.426843)
Stddev: 0.00001 -> 0.00001: 1.1267x larger (N = 500)

Running 'query_complex_filter' benchmark ...
Min: 0.000039 -> 0.000034: 1.1527x faster
Avg: 0.000041 -> 0.000037: 1.1091x faster
Significant (t=13.582718)
Stddev: 0.00000 -> 0.00001: 1.5373x larger (N = 500)

Running 'query_filter' benchmark ...
Min: 0.000127 -> 0.000112: 1.1288x faster
Avg: 0.000133 -> 0.000119: 1.1228x faster
Significant (t=22.727829)
Stddev: 0.00001 -> 0.00001: 1.1771x smaller (N = 500)

Running 'template_render_simple' benchmark ...
Min: 0.000030 -> 0.000024: 1.2405x faster
Avg: 0.000035 -> 0.000029: 1.2042x faster
Not significant
Stddev: 0.00007 -> 0.00005: 1.3190x smaller (N = 500)

Running 'default_middleware' benchmark ...
Min: -0.000047 -> -0.000054: 0.8624x faster
Avg: 0.000017 -> 0.000017: 1.0032x faster
Not significant
Stddev: 0.00037 -> 0.00037: 1.0091x larger (N = 500)

Running 'query_annotate' benchmark ...
Min: 0.000186 -> 0.000162: 1.1505x faster
Avg: 0.000207 -> 0.000178: 1.1660x faster
Significant (t=16.516089)
Stddev: 0.00003 -> 0.00003: 1.1403x smaller (N = 500)

Running 'raw_sql' benchmark ...
Min: 0.000015 -> 0.000013: 1.1070x faster
Avg: 0.000017 -> 0.000014: 1.1676x faster
Significant (t=13.598519)
Stddev: 0.00000 -> 0.00000: 2.3503x smaller (N = 500)

Running 'url_resolve_flat' benchmark ...
Min: 0.056378 -> 0.044772: 1.2592x faster
Avg: 0.058268 -> 0.046656: 1.2489x faster
Significant (t=197.176590)
Stddev: 0.00121 -> 0.00051: 2.3665x smaller (N = 500)

Running 'l10n_render' benchmark ...
Min: 0.001097 -> 0.000727: 1.5092x faster
Avg: 0.001160 -> 0.000768: 1.5101x faster
Significant (t=36.971179)
Stddev: 0.00019 -> 0.00014: 1.2946x smaller (N = 500)

Running 'query_count' benchmark ...
Min: 0.000083 -> 0.000073: 1.1302x faster
Avg: 0.000091 -> 0.000079: 1.1640x faster
Significant (t=15.049336)
Stddev: 0.00002 -> 0.00001: 1.6661x smaller (N = 500)

Running 'model_delete' benchmark ...
Min: 0.000123 -> 0.000105: 1.1701x faster
Avg: 0.000135 -> 0.000119: 1.1396x faster
Significant (t=17.781816)
Stddev: 0.00001 -> 0.00002: 1.1990x larger (N = 500)

Running 'query_iterator' benchmark ...
Min: 0.000102 -> 0.000088: 1.1605x faster
Avg: 0.000108 -> 0.000093: 1.1598x faster
Significant (t=23.872009)
Stddev: 0.00001 -> 0.00001: 1.1366x smaller (N = 500)

Running 'template_compilation' benchmark ...
Min: 0.000155 -> 0.000129: 1.2015x faster
Avg: 0.000169 -> 0.000137: 1.2317x faster
Significant (t=6.119618)
Stddev: 0.00009 -> 0.00007: 1.4162x smaller (N = 500)

Running 'query_all_multifield' benchmark ...
Min: 0.014582 -> 0.012509: 1.1658x faster
Avg: 0.015715 -> 0.013337: 1.1783x faster
Significant (t=19.183517)
Stddev: 0.00207 -> 0.00184: 1.1241x smaller (N = 500)

Running 'query_prefetch_related' benchmark ...
Min: 0.014293 -> 0.012157: 1.1758x faster
Avg: 0.015467 -> 0.013276: 1.1650x faster
Significant (t=20.607411)
Stddev: 0.00176 -> 0.00160: 1.0952x smaller (N = 500)

Running 'query_all_converters' benchmark ...
Min: 0.000536 -> 0.000464: 1.1554x faster
Avg: 0.000563 -> 0.000486: 1.1595x faster
Significant (t=38.503433)
Stddev: 0.00004 -> 0.00002: 1.6468x smaller (N = 500)

Running 'query_distinct' benchmark ...
Min: 0.000106 -> 0.000092: 1.1583x faster
Avg: 0.000127 -> 0.000096: 1.3223x faster
Significant (t=27.798102)
Stddev: 0.00002 -> 0.00001: 3.7187x smaller (N = 500)

Running 'query_dates' benchmark ...
Min: 0.000249 -> 0.000209: 1.1953x faster
Avg: 0.000275 -> 0.000228: 1.2056x faster
Significant (t=30.785168)
Stddev: 0.00003 -> 0.00002: 1.0854x smaller (N = 500)

Running 'model_save_existing' benchmark ...
Min: 0.003526 -> 0.003094: 1.1397x faster
Avg: 0.003723 -> 0.003212: 1.1591x faster
Significant (t=47.274918)
Stddev: 0.00018 -> 0.00016: 1.1817x smaller (N = 500)

Running 'query_delete_related' benchmark ...
Min: 0.000120 -> 0.000103: 1.1655x faster
Avg: 0.000132 -> 0.000111: 1.1815x faster
Significant (t=6.428771)
Stddev: 0.00005 -> 0.00004: 1.2149x smaller (N = 500)

Running 'url_reverse' benchmark ...
Min: 0.000062 -> 0.000060: 1.0318x faster
Avg: 0.000072 -> 0.000068: 1.0622x faster
Not significant
Stddev: 0.00006 -> 0.00005: 1.0531x smaller (N = 500)

Running 'query_latest' benchmark ...
Min: 0.000136 -> 0.000118: 1.1454x faster
Avg: 0.000155 -> 0.000129: 1.2008x faster
Significant (t=8.372115)
Stddev: 0.00007 -> 0.00001: 5.1365x smaller (N = 500)

Running 'form_create' benchmark ...
Min: 0.000015 -> 0.000013: 1.2319x faster
Avg: 0.000019 -> 0.000015: 1.2739x faster
Significant (t=4.158080)
Stddev: 0.00002 -> 0.00001: 1.1449x smaller (N = 500)

Running 'query_update' benchmark ...
Min: 0.000047 -> 0.000041: 1.1323x faster
Avg: 0.000052 -> 0.000044: 1.1721x faster
Significant (t=18.470635)
Stddev: 0.00001 -> 0.00000: 1.6104x smaller (N = 500)

Running 'query_in_bulk' benchmark ...
Min: 0.000152 -> 0.000136: 1.1193x faster
Avg: 0.000173 -> 0.000147: 1.1735x faster
Significant (t=16.901845)
Stddev: 0.00003 -> 0.00001: 2.1199x smaller (N = 500)

Running 'url_resolve_nested' benchmark ...
Min: 0.000043 -> 0.000034: 1.2871x faster
Avg: 0.000075 -> 0.000047: 1.6049x faster
Not significant
Stddev: 0.00066 -> 0.00023: 2.8387x smaller (N = 500)

Running 'model_creation' benchmark ...
Min: 0.000077 -> 0.000066: 1.1579x faster
Avg: 0.000088 -> 0.000072: 1.2205x faster
Significant (t=10.514202)
Stddev: 0.00003 -> 0.00001: 3.1410x smaller (N = 500)

Running 'query_order_by' benchmark ...
Min: 0.000135 -> 0.000124: 1.0945x faster
Avg: 0.000145 -> 0.000133: 1.0902x faster
Significant (t=13.574502)
Stddev: 0.00001 -> 0.00001: 1.1586x smaller (N = 500)

Running 'startup' benchmark ...
Skipped: Django 1.9 and later has changed app loading. This benchmark needs fixing anyway.

Running 'form_clean' benchmark ...
Min: 0.000005 -> 0.000003: 1.4696x faster
Avg: 0.000006 -> 0.000004: 1.4931x faster
Significant (t=11.263253)
Stddev: 0.00000 -> 0.00000: 2.2571x smaller (N = 500)

Running 'locale_from_request' benchmark ...
Min: 0.000076 -> 0.000082: 1.0895x slower
Avg: 0.000083 -> 0.000090: 1.0877x slower
Not significant
Stddev: 0.00009 -> 0.00006: 1.6230x smaller (N = 500)

Running 'query_exists' benchmark ...
Min: 0.000243 -> 0.000214: 1.1399x faster
Avg: 0.000262 -> 0.000227: 1.1571x faster
Significant (t=27.797738)
Stddev: 0.00002 -> 0.00002: 1.2601x smaller (N = 500)

Running 'query_values_10000' benchmark ...
Min: 0.005755 -> 0.005269: 1.0923x faster
Avg: 0.006184 -> 0.005587: 1.1067x faster
Significant (t=10.895954)
Stddev: 0.00094 -> 0.00079: 1.1902x smaller (N = 500)

Running 'query_exclude' benchmark ...
Min: 0.000159 -> 0.000141: 1.1256x faster
Avg: 0.000177 -> 0.000151: 1.1741x faster
Significant (t=23.556200)
Stddev: 0.00002 -> 0.00001: 1.8250x smaller (N = 500)

Running 'query_raw' benchmark ...
Min: 0.005619 -> 0.004860: 1.1562x faster
Avg: 0.006181 -> 0.005041: 1.2263x faster
Significant (t=18.008590)
Stddev: 0.00121 -> 0.00074: 1.6376x smaller (N = 500)

Running 'url_resolve' benchmark ...
Min: 0.004666 -> 0.004233: 1.1023x faster
Avg: 0.004920 -> 0.004347: 1.1318x faster
Significant (t=24.865249)
Stddev: 0.00049 -> 0.00016: 3.1507x smaller (N = 500)

Running 'model_save_new' benchmark ...
Min: 0.003420 -> 0.003105: 1.1014x faster
Avg: 0.003610 -> 0.003217: 1.1221x faster
Significant (t=42.956103)
Stddev: 0.00017 -> 0.00011: 1.6304x smaller (N = 500)

Running 'query_all' benchmark ...
Min: 0.008101 -> 0.007077: 1.1447x faster
Avg: 0.009006 -> 0.007936: 1.1348x faster
Significant (t=9.981534)
Stddev: 0.00171 -> 0.00168: 1.0215x smaller (N = 500)

Django 4.2.6

• Djangobench 0.10.0

• Django 4.0.10

• Python 3.8.18 vs Python 3.11.5

joel@METABOX:~/djangobench/django$ djangobench --vcs=none --control=. --experiment=. --control-python=/home/joel/djangobench/py38/bin/python --experiment-python=/home/joel/djangobench/py311/bin/python -r /home/joel/djangobench/results -t 500
Running all benchmarks
Recording data to '/home/joel/djangobench/results'
Control: Django 4.2.6 (in .)
Experiment: Django 4.2.6 (in .)

Running 'multi_value_dict' benchmark ...
Min: -0.000004 -> 0.000013: -3.0336x slower
Avg: 0.000182 -> 0.000133: 1.3680x faster
Significant (t=9.151616)
Stddev: 0.00010 -> 0.00007: 1.3826x smaller (N = 500)

Running 'query_values' benchmark ...
Min: 0.000082 -> 0.000072: 1.1485x faster
Avg: 0.000086 -> 0.000075: 1.1462x faster
Significant (t=30.114973)
Stddev: 0.00001 -> 0.00001: 1.0258x larger (N = 500)

Running 'query_delete' benchmark ...
Min: 0.000080 -> 0.000071: 1.1169x faster
Avg: 0.000086 -> 0.000077: 1.1088x faster
Significant (t=13.459411)
Stddev: 0.00001 -> 0.00001: 1.0008x smaller (N = 500)

Running 'query_select_related' benchmark ...
Min: 0.016889 -> 0.013513: 1.2498x faster
Avg: 0.018370 -> 0.013885: 1.3230x faster
Significant (t=48.921967)
Stddev: 0.00196 -> 0.00061: 3.2174x smaller (N = 500)

Running 'query_aggregate' benchmark ...
Min: 0.000167 -> 0.000153: 1.0904x faster
Avg: 0.000182 -> 0.000165: 1.1029x faster
Significant (t=12.685517)
Stddev: 0.00002 -> 0.00002: 1.3019x smaller (N = 500)

Running 'query_raw_deferred' benchmark ...
Min: 0.004160 -> 0.003674: 1.1323x faster
Avg: 0.004596 -> 0.003888: 1.1820x faster
Significant (t=11.504156)
Stddev: 0.00117 -> 0.00073: 1.5957x smaller (N = 500)

Running 'query_get_or_create' benchmark ...
Min: 0.000421 -> 0.000356: 1.1823x faster
Avg: 0.000470 -> 0.000392: 1.2011x faster
Significant (t=14.613017)
Stddev: 0.00008 -> 0.00009: 1.0954x larger (N = 500)

Running 'query_values_list' benchmark ...
Min: 0.000080 -> 0.000070: 1.1395x faster
Avg: 0.000085 -> 0.000075: 1.1202x faster
Significant (t=20.300988)
Stddev: 0.00001 -> 0.00001: 1.0537x smaller (N = 500)

Running 'url_resolve_flat_i18n_off' benchmark ...
Min: 0.056031 -> 0.045854: 1.2219x faster
Avg: 0.057048 -> 0.048370: 1.1794x faster
Significant (t=106.668460)
Stddev: 0.00117 -> 0.00139: 1.1819x larger (N = 500)

Running 'qs_filter_chaining' benchmark ...
Min: 0.000247 -> 0.000205: 1.2080x faster
Avg: 0.000267 -> 0.000219: 1.2211x faster
Significant (t=38.507950)
Stddev: 0.00002 -> 0.00002: 1.0252x larger (N = 500)

Running 'template_render' benchmark ...
Min: 0.000956 -> 0.000761: 1.2550x faster
Avg: 0.001061 -> 0.000862: 1.2302x faster
Significant (t=6.128572)
Stddev: 0.00052 -> 0.00051: 1.0109x smaller (N = 500)

Running 'query_get' benchmark ...
Min: 0.000268 -> 0.000235: 1.1388x faster
Avg: 0.000293 -> 0.000256: 1.1411x faster
Significant (t=24.002331)
Stddev: 0.00002 -> 0.00003: 1.2917x larger (N = 500)

Running 'query_none' benchmark ...
Min: 0.000055 -> 0.000050: 1.1079x faster
Avg: 0.000061 -> 0.000055: 1.1183x faster
Significant (t=3.149707)
Stddev: 0.00003 -> 0.00004: 1.3162x larger (N = 500)

Running 'query_complex_filter' benchmark ...
Min: 0.000040 -> 0.000034: 1.1777x faster
Avg: 0.000042 -> 0.000038: 1.1267x faster
Significant (t=15.246074)
Stddev: 0.00000 -> 0.00001: 1.5250x larger (N = 500)

Running 'query_filter' benchmark ...
Min: 0.000131 -> 0.000116: 1.1288x faster
Avg: 0.000139 -> 0.000127: 1.0907x faster
Significant (t=14.448319)
Stddev: 0.00001 -> 0.00001: 1.2281x larger (N = 500)

Running 'template_render_simple' benchmark ...
Min: 0.000031 -> 0.000024: 1.2650x faster
Avg: 0.000037 -> 0.000029: 1.2895x faster
Significant (t=2.094800)
Stddev: 0.00007 -> 0.00005: 1.3630x smaller (N = 500)

Running 'default_middleware' benchmark ...
Min: -0.000037 -> -0.000060: 0.6180x faster
Avg: 0.000001 -> 0.000001: 1.0056x slower
Not significant
Stddev: 0.00002 -> 0.00002: 1.0915x smaller (N = 500)

Running 'query_annotate' benchmark ...
Min: 0.000192 -> 0.000173: 1.1122x faster
Avg: 0.000206 -> 0.000185: 1.1134x faster
Significant (t=17.849733)
Stddev: 0.00002 -> 0.00002: 1.0456x smaller (N = 500)

Running 'raw_sql' benchmark ...
Min: 0.000013 -> 0.000012: 1.0839x faster
Avg: 0.000015 -> 0.000014: 1.0882x faster
Significant (t=4.252084)
Stddev: 0.00001 -> 0.00000: 1.5868x smaller (N = 500)

Running 'url_resolve_flat' benchmark ...
Min: 0.055540 -> 0.046018: 1.2069x faster
Avg: 0.058030 -> 0.048408: 1.1988x faster
Significant (t=98.852976)
Stddev: 0.00157 -> 0.00151: 1.0444x smaller (N = 500)

Running 'l10n_render' benchmark ...
Min: 0.001604 -> 0.001253: 1.2797x faster
Avg: 0.001684 -> 0.001304: 1.2918x faster
Significant (t=37.535402)
Stddev: 0.00017 -> 0.00015: 1.1476x smaller (N = 500)

Running 'query_count' benchmark ...
Min: 0.000176 -> 0.000165: 1.0631x faster
Avg: 0.000189 -> 0.000176: 1.0755x faster
Significant (t=12.229046)
Stddev: 0.00002 -> 0.00002: 1.0395x larger (N = 500)

Running 'model_delete' benchmark ...
Min: 0.000122 -> 0.000104: 1.1743x faster
Avg: 0.000152 -> 0.000115: 1.3227x faster
Significant (t=19.812953)
Stddev: 0.00004 -> 0.00001: 2.6554x smaller (N = 500)

Running 'query_iterator' benchmark ...
Min: 0.000108 -> 0.000094: 1.1518x faster
Avg: 0.000119 -> 0.000098: 1.2203x faster
Significant (t=21.984884)
Stddev: 0.00002 -> 0.00001: 2.7750x smaller (N = 500)

Running 'template_compilation' benchmark ...
Min: 0.000164 -> 0.000148: 1.1034x faster
Avg: 0.000184 -> 0.000162: 1.1386x faster
Significant (t=4.665298)
Stddev: 0.00008 -> 0.00007: 1.2952x smaller (N = 500)

Running 'query_all_multifield' benchmark ...
Min: 0.014802 -> 0.012188: 1.2144x faster
Avg: 0.016029 -> 0.013294: 1.2057x faster
Significant (t=21.516971)
Stddev: 0.00210 -> 0.00191: 1.0984x smaller (N = 500)

Running 'query_prefetch_related' benchmark ...
Min: 0.013401 -> 0.011583: 1.1569x faster
Avg: 0.014822 -> 0.013366: 1.1090x faster
Significant (t=11.422100)
Stddev: 0.00170 -> 0.00229: 1.3410x larger (N = 500)

Running 'query_all_converters' benchmark ...
Min: 0.000499 -> 0.000429: 1.1618x faster
Avg: 0.000531 -> 0.000455: 1.1670x faster
Significant (t=42.716720)
Stddev: 0.00002 -> 0.00003: 1.5394x larger (N = 500)

Running 'query_distinct' benchmark ...
Min: 0.000108 -> 0.000095: 1.1397x faster
Avg: 0.000116 -> 0.000100: 1.1646x faster
Significant (t=25.915629)
Stddev: 0.00001 -> 0.00001: 1.1204x larger (N = 500)

Running 'query_dates' benchmark ...
Min: 0.000333 -> 0.000290: 1.1490x faster
Avg: 0.000365 -> 0.000326: 1.1207x faster
Significant (t=18.213858)
Stddev: 0.00003 -> 0.00003: 1.0118x larger (N = 500)

Running 'model_save_existing' benchmark ...
Min: 0.003455 -> 0.003081: 1.1215x faster
Avg: 0.003764 -> 0.003326: 1.1316x faster
Significant (t=32.229651)
Stddev: 0.00023 -> 0.00020: 1.1398x smaller (N = 500)

Running 'query_delete_related' benchmark ...
Min: 0.000122 -> 0.000102: 1.1946x faster
Avg: 0.000131 -> 0.000113: 1.1564x faster
Significant (t=5.027485)
Stddev: 0.00005 -> 0.00006: 1.4129x larger (N = 500)

Running 'url_reverse' benchmark ...
Min: 0.000068 -> 0.000067: 1.0193x faster
Avg: 0.000075 -> 0.000074: 1.0157x faster
Not significant
Stddev: 0.00006 -> 0.00005: 1.1543x smaller (N = 500)

Running 'query_latest' benchmark ...
Min: 0.000147 -> 0.000138: 1.0631x faster
Avg: 0.000167 -> 0.000148: 1.1277x faster
Significant (t=11.353029)
Stddev: 0.00003 -> 0.00002: 1.6091x smaller (N = 500)

Running 'form_create' benchmark ...
Min: 0.000016 -> 0.000013: 1.2659x faster
Avg: 0.000020 -> 0.000015: 1.2770x faster
Significant (t=3.482649)
Stddev: 0.00002 -> 0.00002: 1.0947x larger (N = 500)

Running 'query_update' benchmark ...
Min: 0.000047 -> 0.000043: 1.0971x faster
Avg: 0.000050 -> 0.000046: 1.0691x faster
Significant (t=9.363513)
Stddev: 0.00001 -> 0.00000: 1.2636x smaller (N = 500)

Running 'query_in_bulk' benchmark ...
Min: 0.000157 -> 0.000143: 1.0970x faster
Avg: 0.000178 -> 0.000162: 1.0981x faster
Significant (t=9.031182)
Stddev: 0.00002 -> 0.00003: 1.5173x larger (N = 500)

Running 'url_resolve_nested' benchmark ...
Min: 0.000046 -> 0.000038: 1.2179x faster
Avg: 0.000075 -> 0.000052: 1.4505x faster
Not significant
Stddev: 0.00059 -> 0.00024: 2.4300x smaller (N = 500)

Running 'model_creation' benchmark ...
Min: 0.000071 -> 0.000065: 1.1058x faster
Avg: 0.000079 -> 0.000073: 1.0876x faster
Significant (t=2.786580)
Stddev: 0.00003 -> 0.00004: 1.1518x larger (N = 500)

Running 'query_order_by' benchmark ...
Min: 0.000146 -> 0.000128: 1.1407x faster
Avg: 0.000154 -> 0.000138: 1.1206x faster
Significant (t=14.021341)
Stddev: 0.00002 -> 0.00002: 1.2540x larger (N = 500)

Running 'startup' benchmark ...
Skipped: Django 1.9 and later has changed app loading. This benchmark needs fixing anyway.

Running 'form_clean' benchmark ...
Min: 0.000005 -> 0.000004: 1.4613x faster
Avg: 0.000006 -> 0.000004: 1.3654x faster
Significant (t=12.763128)
Stddev: 0.00000 -> 0.00000: 1.1666x larger (N = 500)

Running 'locale_from_request' benchmark ...
Min: 0.000097 -> 0.000108: 1.1090x slower
Avg: 0.000108 -> 0.000120: 1.1178x slower
Significant (t=-3.057677)
Stddev: 0.00007 -> 0.00006: 1.1186x smaller (N = 500)

Running 'query_exists' benchmark ...
Min: 0.000273 -> 0.000234: 1.1698x faster
Avg: 0.000290 -> 0.000248: 1.1686x faster
Significant (t=39.518859)
Stddev: 0.00002 -> 0.00002: 1.2025x smaller (N = 500)

Running 'query_values_10000' benchmark ...
Min: 0.005601 -> 0.005298: 1.0571x faster
Avg: 0.006023 -> 0.005691: 1.0583x faster
Significant (t=6.167352)
Stddev: 0.00082 -> 0.00088: 1.0752x larger (N = 500)

Running 'query_exclude' benchmark ...
Min: 0.000159 -> 0.000140: 1.1367x faster
Avg: 0.000165 -> 0.000149: 1.1020x faster
Significant (t=19.643154)
Stddev: 0.00001 -> 0.00001: 1.2636x larger (N = 500)

Running 'query_raw' benchmark ...
Min: 0.005764 -> 0.004630: 1.2450x faster
Avg: 0.006169 -> 0.004881: 1.2638x faster
Significant (t=20.996453)
Stddev: 0.00109 -> 0.00083: 1.3105x smaller (N = 500)

Running 'url_resolve' benchmark ...
Min: 0.004928 -> 0.004597: 1.0721x faster
Avg: 0.005217 -> 0.004716: 1.1063x faster
Significant (t=46.893945)
Stddev: 0.00022 -> 0.00010: 2.2192x smaller (N = 500)

Running 'model_save_new' benchmark ...
Min: 0.003404 -> 0.003012: 1.1301x faster
Avg: 0.003494 -> 0.003105: 1.1251x faster
Significant (t=45.888484)
Stddev: 0.00014 -> 0.00013: 1.0298x smaller (N = 500)

Running 'query_all' benchmark ...
Min: 0.007971 -> 0.007085: 1.1250x faster
Avg: 0.009091 -> 0.008147: 1.1159x faster
Significant (t=7.583074)
Stddev: 0.00183 -> 0.00210: 1.1518x larger (N = 500)