Module Submission for developers

Latest revision as of 08:37, 2 August 2012

WARNING: move from doppio to github

The single project repository is being split to provide improved system integrity and cohesion, and re-hosted at github, so the workflow described here is obsolete.Ask on the #lunar IRC channel on Freenode.net for the latest details.

Module Submission for developers

Why and how users should submit modules is described on the Module_Submission page. This page describes what a developer needs to do to handle these submissions. Or rather, it describes what works for me.

I'm no expert by any means and this is not intended to be a comprehensive guide to git. There is just enough information to get started, and to describe my workflow.

Developer access

Before you will be able to save anything back to the central git repository holding the Moonbase, you will obviously need to apply for a username and password on the central machine. You will need to have been a regular, constructive contributor to the #lunar IRC channel on Freenode.net, or the Lunar Mailing-List or have successfully submitted modules in the past.

Cloning your own moonbase.git

Just like other source code management systems, git uses the concepts of a repository and working files. In the past git has allowed the repository and the working files to exist within the same top level directory structure. It seems that this has lead to some problems, so work is under way to separate out the repository and the working files. At the moment certain operations generate warning messages if you try to save to a combined repository and working directory. In the future you will need to have a "bare" repository and a separate working version. This article describes how to set up and use such a future-proof system.

I don't have a public server of my own on which I can make a repository available for sharing, testing and cherry-picking by others. Therefore I use a multi-stage repository and working directory system. If you have a public server, you may be able to work with just one.

I have three repositories: a remote copy of the main Lunar Linux Moonbase repository; a remote working copy; and a local working copy. The first one is optional, but it allows me to share files between the working copies without having to commit them to the main Moonbase repository first.

First of all I login to the main Lunar Linux machine, doppio, using ssh, and set two important global git defaults:

 doppio$  git config --global user.name "Full Name"
 doppio$  git config --global user.email "username@lunar-linux.org"

Then I create the remote "bare" copy of the main Moonbase repository and configure two useful properties:

 doppio$  git clone --bare git://lunar-linux.org/lunar/moonbase.git moonbase.git
 doppio$  git config --get-regexp '^(remote|branch)\.'
 doppio$  git config remote.origin.push master:master

Next I create a remote "working" repository that shadows the remote "bare" one, again on doppio:

 doppio$  git clone moonbase.git workbase.git
 doppio$  cd workbase.git
 doppio$  git config --get-regexp '^(remote|branch)\.'
 doppio$  git config remote.origin.push master:master

The second git config command tells git that the default action for the "push" command is to push changes from the master branch of the current repository to the master branch of the parent repository. Otherwise you need to provide that information on the command line every time.

I have a local "working" repository on my private machine that also shadows the remote "bare" one. I shall call my local machine satellite so that it matches the instructions given in Everyday GIT With 20 Commands Or So but see the Note at the end of the section.

 satellite$  git config --global user.name "Full Name"
 satellite$  git config --global user.email "username@lunar-linux.org"

 satellite$  git clone ssh://username@lunar-linux.org/~username/moonbase.git workbase.git
 satellite$  cd workbase.git
 satellite$  git config --get-regexp '^(remote|branch)\.'
 satellite$  git config remote.origin.push master:master

This repository setup allows the following workflow:

• first ensure that the my central repo is up to date:

 doppio$  cd moonbase.git
 doppio$  git fetch               # note: fetch not pull

• then, either work in the remote working directory:

 doppio$  cd workbase.git
 doppio$  git checkout master
 doppio$  git pull
 doppio$  git checkout -b testing master
 doppio$  ... edit files and commit ...
 doppio$  git checkout master
 doppio$  git merge testing
 doppio$  git branch -D testing
 doppio$  git push

• or work on the local working directory:

 satellite$  cd workbase.git
 satellite$  git checkout master
 satellite$  git pull
 satellite$  git checkout -b testing master
 satellite$  ... edit files and commit ...
 satellite$  git checkout master
 satellite$  git merge testing
 satellite$  git branch -D testing
 satellite$  git push

• Finally I can update the central Moonbase repository, although I'm sure there must be a better way:

 doppio$  cd moonbase.git
 doppio$  git push /var/git/lunar/moonbase.git master

• I can only share files back and forth between the local and remote working repositories using this method. I can't pull from the working directory on satellite to the working directory on doppio because satellite is private and can only call out. I can't push from the working directory on satellite to the working directory on doppio either because it is not a bare repository.

Note that Everyday GIT... recommends having the following line instead:

 satellite$  git config remote.origin.push master:refs/remotes/satellite/master

but this doesn't work with the bare repository system. Or rather, it does something, but I haven't worked out how to use the information in the remote repository.

Reviewing module submissions

There is a neat script on doppio called review that you can use to view and apply the module submission patches directly to the current working copy of moonbase.git. The review script assumes that the submitter has already tested the patch and that if it applies cleanly it will be accepted, otherwise it will be rejected. One really cool feature of review is that the submitter gets the credit in the commit message. Unfortunately, if there are any problems in your working copy of moonbase.git - and there often were in mine - the script can delete the patch.

By default review will work through all of the modules in the submission queue so it's best to specify the module that you are interested in. review will open an editor session containing the module patch file so that you can view the changes, and remove "Subject: " from the subject line and annotate what will become the text of the commit message. After you leave the editor, review checks whether the patch will apply to the current working copy of moonbase.git without problems, suggests what should be done with the patch, and waits for your input. If the patch applies cleanly, the default action is to accept it. You could still reject it for some other reason; because it does not conform to the Module_Guidelines for example. If the patch can't be applied - and that might mean that the submission was valid but that a developer has already updated that module in the moonbase - the action is to reject and another editor session will open so that you can explain why. The two other possible actions are defer and quit.

You can also use /home/engelsman/review.new which gives the added option to thank someone for a submission that was valid but can no longer be applied because another developer has beaten them to it. It's a little less harsh than reject.

 doppio$ cd moonbase.git
 doppio$ git fetch                      # make sure we are up to date
 doppio$ cd ../workbase.git
 doppio$ git pull                       # make sure working copy is also up to date
 doppio$ git checkout -b testing master # create a test branch in case of problems
 doppio$ review submitted_module        # view patch in editor, then accept,reject,defer,quit

 # if accepted:
 doppio$ git checkout master            # still in workbase.git
 doppio$ git merge testing
 doppio$ git branch -D testing
 doppio$ git push
 doppio$ cd ../moonbase.git
 doppio$ git push /var/git/lunar/moonbase.git master

 # else:
 doppio$ git checkout master            # still in workbase.git
 doppio$ git branch -D testing

If all has gone smoothly, git status won't show any added, deleted or modified files. Take care to clean up if there are.

Testing the submission locally before committing

If you want to be really thorough - or paranoid - you should really check that the patch applies and builds on your own machine first. That takes a little more effort, because you will first need to grab the patch file, apply and test locally:

 doppio$  cd moonbase.git
 doppio$  git fetch                      # make sure up to date before pulling working copy

Make sure that your local moonbase is up-to-date before you try dealing with the patch:

 satellite$  cd moonbase.git
 satellite$  git pull
 satellite$  git checkout -b testing master

and then use the method of your choice to grab the patch and apply it:

 satellite$  # method 1
 satellite$  scp username@doppio.lunar-linux.org:/var/anonymous/submissions/submitted_module.patch /tmp
 satellite$  git am /tmp/submitted_module.patch

 satellite$  # method 2
 satellite$  wget -q "http://foo-projects.org/~sofar/queue.php?p=submitted_module" -O - | git am

 satellite$  # method 3
 satellite$  # browse to "http://foo-projects.org/~sofar/queue.php"
 satellite$  # right click on patch and "save link as" /tmp/submitted_module.patch
 satellite$  git am /tmp/submitted_module.patch

 satellite$  # check lin submitted_module works as expected

At this point, if you want the submitter to get a feel-good "Your patch was accepted" message:

 satellite$  git checkout master
 satellite$  git branch -D testing

 doppio$     # use git and review as described earlier

Otherwise, just continue with the usual git workflow:

 satellite$  git checkout master
 satellite$  git merge testing
 satellite$  git branch -D testing
 satellite$  git push

 doppio$  cd moonbase.git
 doppio$  git merge satellite/testing    # see git config tip earlier
 doppio$  git push /var/git/lunar/moonbase.git master
 doppio$  review                         # patch already applied, so use review thank option

As mentioned earlier, run git status, tidy up if required, and run git pull again.

Is there a better way?

I'm an apprentice rather than a wizard level developer, and I've just about understood git-101, but diagnosing git problems makes my head hurt.

So if you can see a better way of doing things, please add a section at the bottom and describe your workflow.