Using git to develop against subversion repositories

A lot of people have written about using git and why it is in their opinion the best thing since sliced bread or even worse than a stinking turd in your bed....

Go read those if you're interested in yet-another-opinion about sourcecode management :-)

I'm interested in developing with git against a subversion repository which holds the main body of code for whatever reason: company politics, historic reasons, need for a centralized repository, whatever.

This is in fact very simple with the git svn wrappers, it's trivial to create a git-repository which is based on a subversion repository AND stays in sync all the time as well.

Please read: http://www.viget.com/extend/effectively-using-git-with-subversion/ for a nice overview.

What is not much covered is setting up a git repository based on a non-standard subversion layout. I searched around on the net using google and couldn't find a good description.

And guess what I had :-(

I've setup a git repository for a subversion repository which had a non-standard layout for historic reasons. It was way too much trouble to reorganize the subversion repository so I decided to make the git repository match the subversion repos.

The subversion is layed out like this:

/
/trunk
/stable
/branches/virtual
/branches/redesign
/josephs_stuff
/tryout1
/tryout2

Basically the stable branch was started not inside a branches subdir as is standard but on the root of the repository. Later on someone realized that a branches subdir was a good thing to have and added it.

Ideally I'd like to have all branches that are relevant to me as a branch inside git.

The branches I'm interested in are: trunk, stable, virtual and redesign, the rest is just old cruft, left there for no specific good reasons.

I've inited the git repos with the following command:

git svn init --trunk trunk https://svn.server.not/svn/ git-repos

Afterwards I needed to add the other three branches, you need to edit you git config file for this which you can find in:

git-repos/.git/config

The relevant part of my git config looks like this:

[svn-remote "svn"]
url = https://svn.server.not/svn/
fetch = trunk:refs/remotes/trunk
fetch = stable:refs/remotes/stable
fetch = branches/virtual:refs/remotes/virtual
fetch = branches/redesign:refs/remotes/redesign

After saving my git config it's time to actually pull in the subversion content:

git svn fetch

That's it.... now you have a fully functional git repository with branches for the different "branches" in your old and crufty subversion repository.

The format appears to be:

url = $SVN_BASE_URL
fetch = $SVN_DIR:refs/remotes/$GITBRANCHNAME

I'd think twice before I'd deviate the git branch-names from the subversion branch names.

Happy hacking.

public and private ssh keys in ldap

Marlon did some great work on setting up a ldap servercluster for our serverpark at hyves.

We want to use that ldap repository to stick all user-accounts in so we have a better control over who gets to access what.

In order to make that happen we need both the public AND the private key for ssh access in our ldap server.

Getting the public keys in ldap proved fairly trivial, head over to the openssh-lpk project on code.google

Applying the patches is easy if you use Gentoo:

USE="ldap" emerge openssh

will do the trick, your milage may vary on other distributions :-)

Getting the private keys in ldap is completely undocumented....

Here's how you do it:

Add the following to the schema provided by the openssh-lpk guys:

attributetype ( 1.3.6.1.4.1.24552.500.1.1.1.14 NAME 'sshPrivateKey'
  DESC 'OPTIONAL: OpenSSH Private key'
  EQUALITY octetStringMatch
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )

objectclass ( 1.3.6.1.4.1.24552.500.1.1.2.1 NAME 'ldapPrivateKey' SUP top AUXILIARY
   DESC 'OPTIONAL: OpenSSH LPK objectclass'
   MAY ( sshPrivateKey $ uid )
)

Now you need to load the public and private key into the ldap server.
The public key again is easy: It's a single string with no linebreaks in it, any tool will do.
But the private key is slightly more involved, the first couple of lines from my private key are:

-----BEGIN DSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,A683EAADB7A65E5C

o+sletw5nmOZN1Hu62HSIRMaMsqWWDSqbxrRKu5zfk/rXWKDzIU9ULSQ3giGmCRx

You can't load that with ldapmodify into your ldap server because it will mistake the content of your key for ldap attributes. I tried loading it through the webinterface (phpLDAPadmin) but that didn't work either.

The solution is specified in the RFC, you need to encode the key in base64.

base64 -w0 <> yourprivatekey_base64

After that it's trivial to load it into your ldapserver, use whatever tool you like.

Here's a sample ldif file for ldapmodify:

dn: cn=your common name,ou=your department,dc=domain,dc=tld
changetype: modify
replace: sshPrivateKey
sshPrivateKey: LS0tLS1CRUdJTiBEU0EgUFJJVkFURSBLRVktLS0tLQpQcm9jLVR5cGU6IDQsRU5DUllQVEVECkRFSy1JbmZvOiBERVMtRURFMy1DQkMs

Once you have loaded the private and public keys in ldap you need a script to install them in the homedir of your server.

We settled on a combination of autodir and puppet.

Autodir will create the homedir for the user once he logs in for the first time.

We populate the skel directory with puppet, the bash_profile file below takes care of pulling in the keys into the .ssh directory.

# /etc/skel/.bash_profile

LOGNAME=$(whoami)
NUMSESSIONS=$(who |grep ${LOGNAME} |wc -l)
HOST=$(hostname)
[[ -f ~/.firstrun ]] && source ~/.firstrun

if [[ "${FIRSTRUN}" == "YES" ]]
then
# Pick up private keys from ldap
echo "This is a firsttime run"
echo "I need to import your ssh-keys from the ldap server"
echo "Please provide the ldap password when asked"
# Find DN for user
LDAP_DN=$(ldapsearch -LLL -ZZ "(uid=$LOGNAME)" dn | cut -f2 -d:)
ldapsearch -LLL -ZZ -F~/.ssh/ -T ~/.ssh/ -D"${LDAP_DN}" -W -x -tt "(uid=${LOGNAME})" sshPublicKey sshPrivateKey
echo "Decoding your private key"
base64 -w0 -d ~/.ssh/ldapsearch-sshPrivateKey-* > ~/.ssh/id_key
echo "Adding your public key"
mv ~/.ssh/ldapsearch-sshPublicKey-* ~/.ssh/id_key.pub
echo "Cleaning up temporary files"
rm ~/.ssh/ldapsearch-*
sed -i -e 's!YES!NO!g' ~/.firstrun
echo "Your ssh keys have been setup"
fi

# only check if this is the first login
if [ "${NUMSESSIONS}" = "1" ]
then
# cleaning up possible old sessions
/usr/bin/keychain --clear -q

#keychain
echo -e "\e[32;1mAdding private keys to keychain\e[m"

for i in $(find ~/.ssh |grep id |grep -v pub)
do
  /usr/bin/keychain -q ${i}
done
fi

# load ssh agent via keychain
source ~/.keychain/$(hostname)-sh

[[ -f /etc/profile.d/bash-completion ]] && source /etc/profile.d/bash-completion

Keychain is a useful tool written by Daniel Robbins, the original Gentoo architect. It loads your keys into ssh-agent and exports the relevant variables into your shell environment, the net effect is that you have to type your passphrase once.
You can find his current homepage here

Maybe this is helpful for someone trying to do something similar, if you need more detail you can always try emailing me :-)

Ramon

P.S> No, the above script is not production ready, it lacks error checking and support for multiple keys.