Author Topic: Bringing over large svn patches to our git repo  (Read 2550 times)

0 Members and 1 Guest are viewing this topic.

Offline The E

  • He's Ebeneezer Goode
  • Global Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Bringing over large svn patches to our git repo
Yesterday, Karajorma had a bit of a problem. His work on sexp container classes was locked up in a patch file against the last revision of our SVN, and none of the solutions we found for converting an svn patch into a git patch actually worked (If you know how to do this, please mention it!).

I then volunteered to convert the stuff to git manually. Kara then asked me to document the process I used so that others may learn from it.

So, basically, here's the workflow I used.
  • Create a new branch in git based off of the last revision of the svn. In git, this is represented by this commit.
  • Apply the svn patch to a clean checkout of the svn repo.
  • Copy all files changed by that patch into the git checkout.
  • Commit the changes made through this.
  • Merge the newly created branch with the current master in the git repo.
  • Resolve any merge errors.
  • Try to compile, fix any compilation errors. In this case, errors were caused by Kara's use of SCP_hash_map, a container that has since been deprecated and replaced with SCP_unordered_map.
  • Commit any changes made.

And that's it.
Let there be light
Let there be moon
Let there be stars and let there be you
Let there be monsters and let there be pain
Let us begin to feel again
--Devin Townsend, Genesis

 

Offline Goober5000

  • HLP Loremaster
  • Administrator
  • 214
    • Goober5000 Productions
Re: Bringing over large svn patches to our git repo
Good post. You may want to merge it into the master sticky thread.

(Incidentally, the master sticky should also be updated with git instructions at some point.)

 

Offline niffiwan

  • 211
  • Eluder Class
Re: Bringing over large svn patches to our git repo
Er, neatly highlighting one issue of documentation on the forum, only the original poster (or an admin) can update a post.  I much prefer the wiki for this sort of doco, as anyone sufficiently motivated can make the updates (you can see that almost every reference to SVN on the wiki has already been updated :D)

</soapbox>
Creating a fs2_open.log | Red Alert Bug = Hex Edit | MediaVPs 2014: Bigger HUD gauges | 32bit libs for 64bit Ubuntu
----
Debian Packages (testing/unstable): Freespace2 | wxLauncher
----
m|m: I think I'm suffering from Stockholm syndrome. Bmpman is starting to make sense and it's actually written reasonably well...

 

Offline Goober5000

  • HLP Loremaster
  • Administrator
  • 214
    • Goober5000 Productions
Re: Bringing over large svn patches to our git repo
But The E, being a mod of the SCP forum, has the ability to edit posts -- n'est-ce pas?

 

Offline niffiwan

  • 211
  • Eluder Class
Re: Bringing over large svn patches to our git repo
True; but OP+Admins+Mods is still less than "all Wiki accounts"!
Creating a fs2_open.log | Red Alert Bug = Hex Edit | MediaVPs 2014: Bigger HUD gauges | 32bit libs for 64bit Ubuntu
----
Debian Packages (testing/unstable): Freespace2 | wxLauncher
----
m|m: I think I'm suffering from Stockholm syndrome. Bmpman is starting to make sense and it's actually written reasonably well...

 

Offline Jackho

  • 25
  • Gnoti seauton ...
    • Downloads an al
Re: Bringing over large svn patches to our git repo
Hi,
You might find this useful (or not):

Converting a Subversion repository to Git
Converting SVN Commits to Git Patches

Just my 2 cents
A+
Jack H. : "Well... There's much more to check than six..."

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: Bringing over large svn patches to our git repo
I'm still not sure why manually copying the files should have been necessary as opposed to just applying the patch directly to the git repo at the corresponding git hash that matches the svn revision.  The GNU patch utility should handle it, barring maybe some whitespace corrections.
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline The E

  • He's Ebeneezer Goode
  • Global Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: Bringing over large svn patches to our git repo
TRied that, didn't work. I mean, it could just be me not knowing all the right commandline incantations for patch, and I am very definitely interested in a less janky way to do this.
Let there be light
Let there be moon
Let there be stars and let there be you
Let there be monsters and let there be pain
Let us begin to feel again
--Devin Townsend, Genesis

 

Offline niffiwan

  • 211
  • Eluder Class
Re: Bringing over large svn patches to our git repo
I gave it a quick attempt, "git apply --verbose" couldn't find where to apply a heap of chunks, GNU/Linux "patch --binary" managed to apply everything except the MSVC project changes.  I'm guessing there's some whitespace / line ending errors in the patch somewhere, but I didn't continue to look because I've previously found errors in the MSVC project files practically impossible to resolve without serious hacking of the diff.  Or applying by hand, in either case the method outlined above is an easier way to do it.
Creating a fs2_open.log | Red Alert Bug = Hex Edit | MediaVPs 2014: Bigger HUD gauges | 32bit libs for 64bit Ubuntu
----
Debian Packages (testing/unstable): Freespace2 | wxLauncher
----
m|m: I think I'm suffering from Stockholm syndrome. Bmpman is starting to make sense and it's actually written reasonably well...

 

Offline Bobboau

  • Just a MODern kinda guy
    Just MODerately cool
    And MODest too
  • 213
Re: Bringing over large svn patches to our git repo
do you still have your svn repos around? could you apply the patch to svn, to get the full file, then just copy/paste the file into the git repo of the same version as the oldest svn and commit?
Bobboau, bringing you products that work... in theory
learn to use PCS
creator of the ProXimus Procedural Texture and Effect Generator
My latest build of PCS2, get it while it's hot!
PCS 2.0.3


DEUTERONOMY 22:11
Thou shalt not wear a garment of diverse sorts, [as] of woollen and linen together

 

Offline AdmiralRalwood

  • 211
  • The Cthulhu programmer himself!
    • Skype
    • Steam
    • Twitter
Re: Bringing over large svn patches to our git repo
do you still have your svn repos around? could you apply the patch to svn, to get the full file, then just copy/paste the file into the git repo of the same version as the oldest svn and commit?
Might want to re-read the first post...
Ph'nglui mglw'nafh Codethulhu GitHub wgah'nagl fhtagn.

schrödinbug (noun) - a bug that manifests itself in running software after a programmer notices that the code should never have worked in the first place.

When you gaze long into BMPMAN, BMPMAN also gazes into you.

"I am one of the best FREDders on Earth" -General Battuta

<Aesaar> literary criticism is vladimir putin

<MageKing17> "There's probably a reason the code is the way it is" is a very dangerous line of thought. :P
<MageKing17> Because the "reason" often turns out to be "nobody noticed it was wrong".
(the very next day)
<MageKing17> this ****ing code did it to me again
<MageKing17> "That doesn't really make sense to me, but I'll assume it was being done for a reason."
<MageKing17> **** ME
<MageKing17> THE REASON IS PEOPLE ARE STUPID
<MageKing17> ESPECIALLY ME

<MageKing17> God damn, I do not understand how this is breaking.
<MageKing17> Everything points to "this should work fine", and yet it's clearly not working.
<MjnMixael> 2 hours later... "God damn, how did this ever work at all?!"
(...)
<MageKing17> so
<MageKing17> more than two hours
<MageKing17> but once again we have reached the inevitable conclusion
<MageKing17> How did this code ever work in the first place!?

<@The_E> Welcome to OpenGL, where standards compliance is optional, and error reporting inconsistent

<MageKing17> It was all working perfectly until I actually tried it on an actual mission.

<IronWorks> I am useful for FSO stuff again. This is a red-letter day!
* z64555 erases "Thursday" and rewrites it in red ink

<MageKing17> TIL the entire homing code is held up by shoestrings and duct tape, basically.

 

Offline Zacam

  • Magnificent Bastard
  • Administrator
  • 211
  • I go Sledge-O-Matic on Spammers
    • Minecraft
    • Steam
    • Twitter
    • ModDB Feature
Re: Bringing over large svn patches to our git repo
My process was (and this was a long while ago):

Got a .diff file. Manually made the changes to SVN to generate a .patch file. Compared the two in order to note the differences as a template, used that format to inter-convert either/or to the other.

I could probably whip up a small something that could manage conversion, but are there really that many outstanding SVN .patch files that this would be needed for?

I'm also more than happy to volunteer converting one to the other (in either direction).
Report MediaVP issues, now on the MediaVP Mantis! Read all about it Here!
Talk with the FSU on #SCP-FSU Talk with the SCP on #SCP
"If you can keep a level head in all this confusion, you just don't understand the situation"

¤[D+¬>

[08/01 16:53:11] <sigtau> EveningTea: I have decided that I am a 32-bit registerkin.  Pronouns are eax, ebx, ecx, edx.
[08/01 16:53:31] <EveningTea> dhauidahh
[08/01 16:53:32] <EveningTea> sak
[08/01 16:53:40] * EveningTea froths at the mouth
[08/01 16:53:40] <sigtau> i broke him, boys

 

Offline jr2

  • The Mail Man
  • 212
  • It's prounounced jayartoo 0x6A7232
    • Steam
Re: Bringing over large svn patches to our git repo
@Zacam if you do make something, not sure if this is any help, but here:

Quote from: https://jbowes.wordpress.com/2009/06/23/converting-svn-commits-to-git-patches/

Code: [Select]
#!/usr/bin/python
#
# svnrev2git.py - Convert an SVN revsion to a Git patch.
#
# Author: James Bowes <[email protected]>
#
# Usage:
#   $> cd my-svn-repo
#   $> python svnrev2git.py [AUTHORS_FILE] [REV_RANGE | REVSION [REVISION..]]
#
#   AUTHORS_FILE - a CSV of  svn username, full name, email
#   REV_RANGE - an svn revision range, like 100-700
#   REVISION - a single svn revision
#
#   You may specify either a revision range, or a series of individual
#   svn revisions
#
# Output:
#   A series of git style patch files, one per svn revision, which can then be
#   applied with 'git am'
#
# Why use this instead of 'git svn'?
#   I had done a large repo conversion via git svn where we wanted no downtime
#   for the switchover. After removing the git svn specific info from our git
#   commits, I used this tool to bring in commits from svn, keeping svn and git
#   in sync, until we were ready to switch.

import sys
import commands

def svnlog_to_gitlog(authors, svnlog):

    lines = svnlog.split("\n  lines = lines[1:-1]

    metainfo = lines[0].split(" | ")
    subject = lines[2]
    description = lines[3:]

    author = metainfo[1]

    day = metainfo[2].split("(")[1][:-1]
    time = metainfo[2].split(" ")[1]
    offset = metainfo[2].split(" ")[2]

    gitlog = []
    gitlog += ["From: %s <%s>" % authors[author]]
    gitlog += ["Date: %s %s %s" % (day, time, offset)]
    gitlog += ["Subject: [PATCH] %s" % subject]
    gitlog += [""]
    gitlog += description
    gitlog += [""]

    return '\n'.join(gitlog)

def svndiff_to_gitdiff(svndiff):
    lines = svndiff.split("\n   gitdiff = []
    for line in lines:
        if line.startswith("--- "):
            gitdiff.append("--- a/" + line[4:])
        elif line.startswith("+++ "):
            gitdiff.append("+++ b/" + line[4:])
        else:
            gitdiff.append(line)

    return '\n'.join(gitdiff)

def make_patch(authors, rev):
    out = commands.getoutput("svn log -c %s ." % rev)

    if len(out.split("\n 2:
        print "skipping r%s" % rev
        return

    patch = open(rev + ".patch", 'w')
    patch.write(svnlog_to_gitlog(authors, out))
    patch.write("---\n\n   out = commands.getoutput("svn diff -c %s ." % rev)
    patch.write(svndiff_to_gitdiff(out))

    patch.write("\n---\n  patch.write("svnrev2git.py\n   patch.close()
    print "wrote %s.patch" % rev

def main(args):
    author_file = open(args[0])
    authors = {}

    print "loading authors"
    for line in author_file.readlines():
        parts = line.strip().split(", ")
        authors[parts[0]] = (parts[1], parts[2])

    author_file.close()

    revs = args[1:]

    if len(revs) == 1 and '-' in revs[0]:
        start, end = revs[0].split('-')
        start = int(start)
        end = int(end)
        revs = [str(x) for x in range(start, end + 1)]

    for rev in revs:
        make_patch(authors, rev)

if __name__ == "__main__":
    main(sys.argv[1:])

Quote from: https://gist.github.com/neosergio/5979125
Code: [Select]
git diff  --no-prefix --ignore-space-at-eol | sed -e "s/^diff --git [^[:space:]]*/Index:/" -e "s/^index.*/===================================================================/" > changes.patch


If this isn't useful, my apologies.
« Last Edit: June 18, 2015, 03:24:23 am by jr2 »