Testability / portability (and fastly invalidation on devguide)

[JulienPalard]

I implemented some new flags to allow the script to be executed anywhere, which is necessary for me to test it locally:

• --log-directory: To log outside of /var/logs/ when testing locally
• --group: To chgrp to someone else than "docs" when testing locally
• --git: To use git instead of mercurial
• `--skip-cache-invalidation: To avoid calling PURGE on fastly (which are probably just ignored).

Obviously the new options have production-ready default values, so it's again a drop-in replacement: no salt modification needed.

I also resolved the TODO about implementing fastly cache invalidation on the devguide.

CC @ewdurbin

[ewdurbin]

@JulienPalard this appears to have caused issues with the devguide. i'm investigating, but see python/devguide#79

[ewdurbin]

fixed via ff091ee

[JulienPalard]

D'oh sry for this, don't hesitate to HL me on IRC for this kind of situation, I do not so often look at my github and I missed your comment.

[JulienPalard]

Look like the doc is no longer up-to-date, this pull request may have broken something, can someone take a look on what's happening on the server? I have no access. ping @ewdurbin

[ewdurbin]

@JulienPalard taking a look. thanks for the flag.

[ewdurbin]

@JulienPalard here's some info from recent logs: https://gist.github.com/ewdurbin/69a31c2b6844a4275f99f32ae4e32740

I don't see any errors.

[ned-deily]

Looking at the build directory, it appears that the docs are being rebuilt. I think it's just a CDN caching issue. In the log, there is a CDN purge of the devguide trees but I didn't see any for the Python doc set trees, like https://docs.python.org/3.6/, ../3.7, ../2.7/, and the aliases to them ../3/. ../2/, ../dev/ etc etc. Perhaps the whole docs.python.org should be purged regularly? I manually purged a few of the index.html URLs and the contents did update.

[JulienPalard]

@ned-deily Historically only python builds (like /2.7/, /3.5/, ..) were purged. This PR (see title) introduces purge on devguide. We still have to work on aliases, that looks manually done (don't remember if I searched but I think so, and I think I not found them in salt scripts, so they may have been done manually).

Look like this PR broke the cache invalidation for python builds. I typically test this script locally with "--skip-cache-invalidation", so I may have missed a bug about it. I'll investigate soon.