Automating the Github/Forum PR stuff?

[kkremitzki]

I like the idea of using the forum as a place to discuss pull requests, but it's a bit of a pain as it is now. I'd like to explain how I could help.

Github repositories have an option to set up a webhook that fires on the "pull request" event and sends an HTTP POST with an information payload (example here) to a server you specify.

There are several small Python tools that can be used to receive webhooks and call handling scripts (for example here). Once set up, a Python script can be written to parse the result, log in to the forum with a specially created account, and create a thread in the pull request forum, linking the PR, and including the title and text of the PR. Then, the URL of that thread can be passed asynchronously to the same Python script handling the webhook, as a result. Using that URL, the original information from the Github pull request, PyGithub, and a specially created Github account with write privileges on the FreeCAD repository, the Github PR can have a comment added linking the forum thread and locking conversation, fully integrating the forum as the place PRs get discussed and handled.

Going further, this can also be used to handle moving threads in the PR forum upon merge/close as well.

I'd like to help set this up as I'm familiar with this sort of thing. I can even use my own infrastructure, although it would be better of course if it was on a freecadweb.org server.

Before I go firing off my VMs to get a working solution, though, I wanted to see what everyone thought of this idea, especially whoever is actually doing the system administration for the project.

[microelly2]

+1
it's good to have such an interface not only for change request but for changes on models too.

[triplus]

Although it does sound as something that could automate things adding write privileges for FreeCAD GitHub repository to some script likely isn't the most desired solution?

[kkremitzki]

Although it does sound as something that could automate things adding write privileges for FreeCAD GitHub repository to some script likely isn't the most desired solution?

It's definitely something to give some thought to. Here are a few points on the matter.

• 1. The Github access will be through PyGithub. I won't be rolling my own API library. This is a repo with over 1000 stars, so we should hopefully see little to no unexpected behavior here.
  2. The portion of my script will, at first, do almost nothing with PyGithub, only post in and lock the comments on PRs. What's being done with the write-access account should be very transparent.
  3. This script will be generalized and written open sourced since others may find a Github webhooks <-> phpbb integration useful. There will be eyes on the whole code to reduce unexpected behavior.
  4. Most importantly, the end result should ideally be ran on FreeCAD-controlled infrastructure. Then, project secrets like the Github and phpbb account info can be kept in a .gitignore'd environment file sourced by the whole thing. That makes the level of security equivalent to "has filesystem access to the FreeCAD server". At that point, IMO, there's nothing to worry about. Standard best practices for this sort of thing have been followed.

[triplus]

Yes the idea itself is not bad but execution usually comes down to introducing additional maintenance burden. In addition there needs to be a server infrastructure involved and i don't know what is the current level of access our hosting provider offers.

One thing is creating the proposed solution (that is what you offer) and another things is somebody that already has write access to GitHub FreeCAD repository would likely need to understand, manage and maintain the solution it in the future. To do things like regular updates of PyGithub and more.

P.S. Therefore i guess wait first to see if anybody is interested in doing that.

[yorik]

kkremitzki excellent idea! I'm all for it. There could even be magnificent extensions, such as the possibility to automatically archive the forum thread once the PR has been committed

Although it's common, I'm not 100% sure our web hosting server can run python scripts, I'll check that. It's of course always possible to host it somewhere else if needed.

[triplus]

Hi @kkremitzki.

We had a discussion about FreeCAD planet not that long ago and the final outcome was current "web service infrastructure maintainers" are already too stressed out and can't take any additional maintenance burden. Therefore i apologize if i gave you that impression in my answers as i was under such impressions myself (for that to represent a problem).

P.S. Anyway looking forward to see your idea materializing in the future!

[saso]

Well, on the other hand, since it has become more common to just do everything related to pull requests directly on GitHub and since also GitHub has recently add some nice improvements (and seems to be adding more) to work with pull requests (eg code reviews) one could also think if it would not be more easy and simple to stop adding them also in the forum. If things needs to be discuses on the forum, before the merge a new topic can still be started... I would probably go in this direction to simplify things, but it is not my decision to make

What are the benefits to having them all also in the forum?

[kkremitzki]

Hi @kkremitzki.

We had a discussion about FreeCAD planet not that long ago and the final outcome was current "web service infrastructure maintainers" are already too stressed out and can't take any additional maintenance burden. Therefore i apologize if i gave you that impression in my answers as i was under such impressions myself (for that to represent a problem).

P.S. Anyway looking forward to see your idea materializing in the future!

I definitely see where you're coming from; I read about the strain level as well. I would consider myself probably a mid-level Linux sysadmin among other things so I figured that this is actually an area I could help out, starting with this automation idea of mine.

@saso I considered the same. I think GIthub's interface is good for short, code-focused conversations. FreeCAD though is very visually oriented and it seems like phpbb is a better environment for long discussion threads and posts with many images, etc., plus all the accounts that are already here.

@yorik Because it is a small Flask app that doesn't require a database, and will not have much traffic (small json payloads on push, PR, merge, close, etc.), Google App Engine may be a good home. It should fit in the free tier of usage. The downside is time & effort spent messing with GAE.

[yorik]

We have to discuss the PR-on-the-forum thing with Werner, if it's fine for him to only use github it's fine for me too. It's just the two of us who commit at the moment, and it's totally manageable.

The main reason for me was that before github, we had the code on sourceforge. Then sourceforge screwed everything. Then we moved to github. So, how long before github is bought by some commercial prick and screwed too? Or things evolve and we can finally host our own git? Then we'd have to move again. Is is wise to do our stuff only on github then? OTOH, this is just pull requests, it's not some part of history we need to keep...

@yorik Because it is a small Flask app that doesn't require a database, and will not have much traffic (small json payloads on push, PR, merge, close, etc.), Google App Engine may be a good home. It should fit in the free tier of usage. The downside is time & effort spent messing with GAE.

Yes, there are many possibilities. I'd prefer to host as much as possible ourselves, though ( http://www.peppercarrot.com/en/article3 ... -libraries )