Back to the main page.
Bug 3049 - migrate from svn to github
| Status | CLOSED FIXED |
| Reported | 2016-01-19 09:43:00 +0100 |
| Modified | 2016-06-14 16:14:46 +0200 |
| Product: | FieldTrip |
| Component: | core |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Mac OS |
| Importance: | P5 normal |
| Assigned to: | Robert Oostenveld |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | 30653066 |
| See also: | http://bugzilla.fieldtriptoolbox.org/show_bug.cgi?id=2948http://bugzilla.fieldtriptoolbox.org/show_bug.cgi?id=1449 |
Robert Oostenveld - 2016-01-19 09:43:14 +0100
With the small team in Nijmegen and the increasing number of "external" contributions, we should make it easier to integrate the external contributions. Furthermore, we should make it more manageable to develop without committing immediately to the release version. Hence we should move to git, which means that I want to move to github.
Robert Oostenveld - 2016-01-19 09:52:11 +0100
On http://www.fieldtriptoolbox.org/faq/why_is_fieldtrip_maintained_in_svn_and_not_in_git are some details that outline why we use svn. Those are to be kept in mind while migrating. in short: 1 dashboard 2 general access control 3 access control for fieldtrip/external 4 hooks The dashboard should change anyway, since right now it is too late in catching errors (only in the release). It should be possible to test the whole batch of scripts against the pre-release version before merging it with the release version. Probably the automation of the dashboard should be abandoned. Right now it automatically runs batches of test scripts, updates http://fieldtriptoolbox.org/development/dashboard and sends an email to fieldtrip-dev a few times a week. If any, then the future dashboard should be based on something like https://travis-ci.org General access control is not a problem with github. Only a few people (right now only me, but could be extended) would need write/management access to fieldtrip/fieldtrip, most people would work on their own branch and send me pull requests. Specific access control to fieldtrip/external cannot be maintained. We should check documentation and make sure that it is explained how to deal with external code (i.e. make changes upstream). The hooks are the most challenging feature to replicate on github. I'll write a separate comment for that.
Robert Oostenveld - 2016-01-19 10:03:05 +0100
Before I forget: we should also consider how to continue with versioning. Right now ft_version is SVN-aware and we use the rXXXX and the ftp release data as versioning schemes. We could continue with the ftp release dates, but the revision number won't apply any more.
Arjen Stolk - 2016-01-19 19:11:27 +0100
How much longer before this wine-drenched brain needs to acquire github skills?
Vladimir Litvak - 2016-01-19 20:35:59 +0100
(In reply to Arjen Stolk from comment #3) If we are talking about github website it also supports SVN clients. So in principle it should be possible to work as before.
Arjen Stolk - 2016-01-20 04:59:15 +0100
(In reply to Vladimir Litvak from comment #4) That's promising!
Robert Oostenveld - 2016-01-22 10:48:50 +0100
Prior to reimplementing the autosync svn hook, I started cleaning up the auto sync properties. My plan is to make a bash script fieldtrip/bin/synchronize-private.sh that contains all the files that need to be synchronized, plus the logic to sync them. That script could be called by any developer, but would also be called by the github push web hook. After synching by this script, a commit and push would be done. The code for the web hook is https://github.com/fieldtrip/webhook.
Robert Oostenveld - 2016-01-22 11:38:43 +0100
mac011> svn commit bin/ Adding bin Adding bin/synchronize-private.sh Transmitting file data . Committed revision 11100.
Diego Lozano Soldevilla - 2016-01-27 19:31:31 +0100
(In reply to Robert Oostenveld from comment #7) Hi Robert, A simple question: who is allowed to merge pull requests? I made a pull request (https://github.com/fieldtrip/fieldtrip/pull/74) but it seems I don't have permissions to merge. [diego@localhost fieldtrip]$ git push upstream origin Username for 'https://github.com': dieloz Password for 'https://dieloz@github.com': remote: Permission to fieldtrip/fieldtrip.git denied to dieloz. fatal: unable to access 'https://github.com/fieldtrip/fieldtrip.git/': The requested URL returned error: 403 On svn times it was easier ;) (sorry for my low git skills...)
Robert Oostenveld - 2016-01-28 08:31:16 +0100
(In reply to Diego Lozano Soldevilla from comment #8) Hi Diego, At this moment I am the only one that can commit and write to github. All code is still going first to our svn system and from there automatically (under my user account) to github. As soon as the svn server is disabled, I will open write/merge access to github to the core team. I have made good progress with the replacement of the svn hooks by web hooks and expect that all relevant functionality will not require the svn server any more. best Robert
Robert Oostenveld - 2016-01-28 08:57:56 +0100
I just wrote this in an email to the fieldtrip-dev and fieldtrip-bugs list. Those include all people that have an svn account. -------- To facilitate the integration of code contributions from (external) coworkers on FieldTrip and from all its users, we will move away from our locally managed svn server towards github. We already have a presence on https://github.com/fieldtrip/fieldtrip/ for quite some time, bot so far code would always be committed first to svn, and from there (under an administrative account) be pushed to github. In the future the code suggestions can be comittted to github by the core development team, or "pull requests” from all contributors can be made and easily merged by the core team. The organization of the code itself is not going to change. If you still have local changes to your svn copy of FieldTrip, please commit these changes to the svn repository or remember them. Next week Wednesday (3 Feb) I will disable write access to the svn server and enable wite access for the core development team to the github repository. From that moment on all code changes and contributions should be committed to github. --------
Diego Lozano Soldevilla - 2016-01-28 09:54:37 +0100
(In reply to Robert Oostenveld from comment #9) thanks Robert! I'll continue improving my git skills next week ;)
Robert Oostenveld - 2016-02-03 17:52:35 +0100
I have switched the SVN repository to read-only. If you now try to commit, you will see this mac011> svn commit svn: E170001: Commit failed (details follow): svn: E170001: Authorization failed svn: E170001: Your commit message was left in a temporary file: svn: E170001: '/Users/roboos/matlab/fieldtrip/svn-commit.tmp' From now on you will have to work with git. See http://www.fieldtriptoolbox.org/development/git which has a full tutorial.
Robert Oostenveld - 2016-02-03 18:05:12 +0100
regarding 1 dashboard 2 general access control 3 access control for fieldtrip/external 4 hooks 1) this is in practice disabled now, since there will not be any new svn commits. I'll have to reimplement it for git and will probably open a new bug for it. 2) this is now through reviewing pull requests and the development team 3) identical as 2 4) the twitter and gitsync hook are reimplemented (running on a raspberry pi in my basement) I still need to update wiki documentation (mainly delete svn stuff) and I still have to update the ftp and the /home/common releases. Right now, those releases will remain stuck on the latest svn version (revision 11162).
Robert Oostenveld - 2016-02-03 18:06:30 +0100
(In reply to Robert Oostenveld from comment #13) sorry, the last version is r11163 and not r11162.
Robert Oostenveld - 2016-02-04 12:38:52 +0100
(In reply to Robert Oostenveld from comment #14) FYI: I just merged two pull requests (one from Roemer, one from myself) on github into the master branch. The github version is now definitely ahead of svn and there is no going back.
Robert Oostenveld - 2016-02-05 10:28:44 +0100
I have gone through the documentation and updated http://www.fieldtriptoolbox.org/development/cvs (yes, it goes back a long time) http://www.fieldtriptoolbox.org/development/svn http://www.fieldtriptoolbox.org/faq/why_is_fieldtrip_maintained_in_svn_and_not_in_git http://www.fieldtriptoolbox.org/faq/i_would_like_an_automatic_daily_updated_version_of_the_code_can_i_get_access_to_the_cvs_or_snv_server http://www.fieldtriptoolbox.org/development I also posted a news item and message on the mailing list. On the wiki there were the following change logs, which I removed 1004 $ find . -name \*changelog\* ./changelog ./changelog/forward_changelog.txt ./changelog/inverse_changelog.txt ./changelog/forward_changelogrecent.txt ./changelog/forwinv_changelogrecent.txt ./changelog/fileio_changelogrecent.txt ./changelog/forwinv_changelog.txt ./changelog/specest_changelog.txt ./changelog/realtime_changelog.txt ./changelog/changelogrecent.txt ./changelog/fileio_changelog.txt ./changelog/connectivity_changelog.txt ./changelog/preproc_changelog.txt ./changelog/changelog.txt ./changelog/specest_changelogrecent.txt ./changelog/connectivity_changelogrecent.txt ./changelog/multivariate_changelog.txt ./changelog/preproc_changelogrecent.txt ./changelog/multivariate_changelogrecent.txt ./changelog/realtime_changelogrecent.txt ./changelog/inverse_changelogrecent.txt
Robert Oostenveld - 2016-02-05 12:57:37 +0100
since it is a lot of hassle to continue with the modules (i.e. the separate zip format releases for fileio, peer, etc). I have removed the modules directory from the FTP server and disabled the (svn based) cron jobs.
Robert Oostenveld - 2016-02-05 13:09:05 +0100
I have reimplemented the internal release on home/common and the external zip file release on the ftp server. They run as frequently as they used to do. The module zip files are not released any more.
Robert Oostenveld - 2016-02-05 13:10:39 +0100
I have made some more edits on the wiki. I think I am done ... if you disagree or realize that something went wrong, please reopen.
Arjen Stolk - 2016-02-05 16:58:48 +0100
Nice work. Can't wait to 'master' git slang. ;)