Nfo's picture

hello, I have just been asked to update a very old turnkey gitlab to the latest version keeping all the data. I do not know where to start, since I’m not familiar with this platform. Can you help me??

Thank you

Forum: 
Jeremy Davis's picture

Sorry for the length of this post, but hopefully it's helpful!

FWIW, we're in the process of making some radical changes to our GitLab appliance which will make this sort of thing much easier going forward (see this thread if you're at all interested). Unfortunately, none of that will apply to you currently. Although once we have released v15.2 and you have everything up to date on your end, then I would highly recommend that you export the data and import it back into the current appliance. From then on updates should be a breeze!

Previously, we installed GitLab from source. So what you'll need to do, is manually update. There will likely be points along the journey, where you'll need to also update dependencies. If they are installed from Debian repos, you may be better off updating the Debian version, otherwise you may need to remove the Debian package and install from source. That may also required migrating data. Some of those might be ok to do straight up (but no guarantees)? Updating some dependencies too early though may be problematic (e.g. Ruby and Rails).

GitLab is a bit of a beast really. Also over the years since we first released the appliance, we have changed the way bits are installed (e.g. if the required version can be provided by Debian packages, we install from the repos, other times we've needed to install from source). So unfortunately, I can't give you any guaranteed step-by-step. The best I can offer is some initial guidance and attempt to assist you if (probably more likely when) you hit issues... You may also find consulting the history of the build code useful. FWIW, each release we've done is tagged, but we have skipped a lot of releases, so you won't be able to follow our changes exactly (there may be intermediate steps required).

When you say, you're "not familiar with this platform" it's not clear whether you mean you're unfamiliar with TurnKey or GitLab, or both. I'll assume both for now as even if you are familiar with one, my info will hopefully still be of value.

First up, TurnKey is based on Debian. The specific version of TurnKey you have (at least since v12.x and up until the current version v15.x, and likely into the future) directly correlates with a Debian version. I'm guessing that you most likely have v13.x. To double check, run 'turnkey-version'. It will spit out something like this: 'turnkey-gitlab-13.0-wheezy-amd64' where the appliance name is 'gitlab', the TurnKey version is '13.0', the Debian codename is 'wheezy' and the architecture is 'amd64' (i.e. x86_64).

Personally, my recommended first step would be to take a full image/snapshot/copy of your current system. Then worst case scenario, you can start again. I'd also recommend that you take many more full images/snapshots/copies as you progress, so you don't have to redo everything if things go pearshaped. I would also urge you to document everything you do. That way, if you do need to backtrack, you can easy re-run previous commands that were taking you in the right direction.

Beyond that, the only other specific advice I can give is links to the GitLab docs. Here are some ones that jump out as being potentially invaluable:

  • Overview
  • generic update instructions
  • specific version-by-version instructions
  • Sorry that I can't provide anything better or more explicit, but hopefully that's enough to get you going in the right direction. TBH, I'm no expert when it comes to GitLab specifically, nor Ruby on Rails apps (or Ruby at all) generally but I do know TurnKey quite well and have had some experience with GitLab, so please feel free to share any problems you encounter and I'll do my best to assist.

    Good luck!

    Jeremy Davis's picture

    It doesn't go as far back as your version, but there is some documentation for updating GitLab in TurnKey versions v14.0 & v14.1. Whilst it won't be a direct help, it may still be worth a read...

    Nfo's picture

     
    Thank you very much for your help ...

    I have a backup in vmware esxi, I can try and try.
    The issue is that in the company where I work I have been assigned this task and I do not know how to do it since I am not familiar with gitlab or turnkey.

    Currently the version of gitlab that I have is 5.0.2, it is very old and I am also almost spaceless. That's my main problem to make backups locally ...

    Today I will read all the documentation you have provided me and if I have any questions I ask you. In any case I will continue to entangle until I achieve the goal of updating with all the data.

    Thank you
    Jeremy Davis's picture

    TBH, I would recommend that you create a complete copy of your VM (at least the virtual) disk(s)) before you go too far. IIRC the best way to do that is via the "export" feature of VMware (although I'm not really familiar with VMware ESXI).

    Whilst a TKLBAM backup (or other file level backup) is useful, in your case, I would recommend a full backup of the whole machine. That way if you break things, you can really easily get back to where you are. "Snapshots" are also useful too, but if anything happens to the base vHDD, they may still cause troubles.

    When you say you're "out of space", I assume that you mean on this VM itself? I hope so! If that's the case, then by default our VM builds should use LVM. Assuming I'm right, then adding an additional vHDD to your filesystem to increase the free space is really easy. This blog post is really old, but it's still relevant and should still work. Assuming that you have room on the VMware host, I suggest you read through that and extend the filesystem so you have plenty of headroom.

    Having said that, I see further down that you have room for a 10GB backup file, so perhaps you don't really need to extend the filesystem just yet?!

    Nfo's picture

    I have this version: turnkey-gitlab-13.0-wheezy-amd64

    Jeremy Davis's picture

    We no longer have that available for download, so it's really important that you backup the whole VM! (E.g. via a VMware "export" or similar). A TKLBAM backup will be of some assistance, but a full backup of the VM will get you back to where you are now much quicker and easier!

    Nfo's picture

    I need help. I managed to make a copy of all the system data on the hard drive of the turnkey.

    The copy is made with tklbam using the command:
    tklbam-backup --address file: // backup

    the file occupies about 10 Gb. The problem I have is that I do not know how to get it out of the virtual machine. I have tried to load a livecd and copy the file to a usb, to mount the virtual machine to access the files .... the only thing that works for me is to download the file from the virtual machine, but it takes 10 hours and it is too long , although it could be done ...

    I have also tried to create a second hard drive and copy it by ssh to the new hard drive, but it gives me an error in the permissions ... I do not know how to continue.

    Thank you.

    What I want to do is get that backup copy and import it in the latest version of turnkey gitlab .... If I manage to import it, everything would work for me?

    I attached screenshot of the end of the backup ....

    Thanks for your help

    Jeremy Davis's picture

    GitLab is a beast of a thing and has a really rapid development cycle. They make lots of changes on a fairly regular basis. So unfortunately, the only way that I can fully recommend (which I have done; at least in part) is for you to manually upgrade through a number of separate versions. You won't need to upgrade through each individual minor version but there will still be lots of steps.

    To clarify, IMO your ultimate goal should be to get from where you are now (a v5.x GitLab "source install" - backed with MySQL DB) to the current latest version of GitLab - installed via the Omnibus package (which includes a PostgreSQL DB back end), running on a current OS version (ideally TurnKey v15.x/Debian Stretch IMO - but I'm biased).

    Since I started writing this post I've actually changed my mind a little on what might be the best way to go. There are other options, which may be a bit quicker, and involve moving to the Omnibus installer sooner that my earlier thoughts (further down this post - below the first dividing line). I haven't tried them and are not sure what issues you may hit, but it may well be a better option. So if you want to try cutting some corners, then you can try googling for something like "gitlab source install with mysql to omnibus". FWIW, here are a few links I found from a quick google:

  • Gitlab 10.3 source install MySQL to Omnibus Postgres Migration
  • Migration from source installation 8.0.5 MySQL to omnibus 8.0.5 PostgreSQL
  • Migration from Non-omnibus to omnibus(and mysql to postgresql)
  • Upgrading GitLab CE from a 6.9.2 source installation to 10.1.0 omnibus, a novel
  • That last one, actually looks quite good (although I note they're using Ubuntu and already have a PostgreSQL backend). FWIW Ubuntu is based on Debian (just like TurnKey) so most tutorials using Ubuntu are relevant to TurnKey (and Debian in general). However it is important to note that Ubuntu is NOT binary compatible with Debian (TurnKey is). So be a bit careful following Ubuntu tutorials blindly. The main thing to avoid is installing packages using Ubuntu repositories. Otherwise most things should "just work" (or fail without doing any damage).

    TBH, the more I think about it, the more I think that trying to migrate it to Omnibus as soon as is possible, is possibly the best approach. Although like I say, I don't have much experience with that (hence why I didn't think of it sooner). I actually wrote the below before I considered that, but I'll leave it there as it will almost certainly be of value. Even if you don't use all of it.


    To get started on the "step-by-step" upgrade journey, start here to upgrade your GitLab v5.0 to v5.1. Once you are on GitLab v5.1, then you can jump straight to v6.0 via this doc.

    Once you get to v6.0, then you can skip quite a few versions to v7.14 via the doc page that I wrote a while ago (see also the GitLab docs on v6.0-v7.14).

    Then from v7.14, you can jump to v8.0. Then to v8.1 and v8.2. Unfortunately, that's were my documentation ends. :( From there, you'll need to go back to the generic GitLab docs, and it appears that you'll need to go through each version separately. I.e. v8.2 -> v8.3, v8.3 -> v8.4, v8.4 -> v8.5 and so on... (find all the doc pages here).

    Did I mention that this is a big job and is going to require a lot of patience and tenacity...?! :(

    Once you get to version 9.3, then you should be able to use the GitLab project "export" feature (as documented here) and migrate your data pretty easily to an "Omnibus" install. As I think I mentioned, that's what I'm currently working on. Once you are on the Omnibus install, then maintenance will be tons easier!

    As noted in the GitLab export/import docs (link above), that feature was introduced in 8.9. I suggest manually going all the way to v9.3 because that is the oldest version available from the Omnibus repository for our Debian Stretch based v15.x appliances (and the upcoming TurnKey v15.2 GitLab appliance I'm working on). However, another option would be to create a new server from our v14.x (Debian Jessie based) Core system (you'll need to download direct from the mirror) and manually install the (v8.9) GitLab Omnibus package yourself on that. It's actually pretty easy to manually do that (automating it, not so easy...). I won'tr bother detailing that yet, but I'm sure you could find the docs yourself, or feel free to hit me up when you are getting close.


    Final word (for now): If you try to migrate to Omnibus sooner that (as per my more recent thoughts, higher up in this post), please share how you go and provide links to any additional resources that you find helpful. It'd be really good to share your first hand knowledge and experience as I'm certain it will help others!

    Good luck!

    Nfo's picture

    Thank you very much for your time.
    Once the initial points are clarified, I will make the scaling that you mention to me. As soon as I have a problem, I consult you ... I hope I am not too heavy.

    On the other hand I am working on a virtual machine where I can do everything, since it is one of tests to try to perform the process. I will make a document with all the steps, perfecting it after getting the migration in the testing machine, doing it in the production machine.

    That document will be shared at the end of the task. I know it's going to be hard and desperate ... but we'll end up getting it fixed.

    A thousand thanks again

    Nfo's picture

    I detail the first steps I have taken.

    1. I need to expand the virtual hard disk. In vmware esxi expanding the disk up to 60 GB. I do it directly with the gparted by booting a livecd from ubuntu. Expanding the partition so that it occupies all the added disk space

    2. By ssh I execute the command df -h to check the space ... this is not assigned, in the virtual machine I have 60gb but in the system I continue with 20gb.
    df -h

    3. I have to correct the partition table:
    fdisk /dev/sda2
    you have to record the changes by entering the w command

    4. Expand the partition:
    lvextend -L + 42G /dev/turnkey/root

    5. expand the partition:
    resize2fs /dev/turnkey/root

    6. We check the status of the partition:
    df -h

    I already have 60gb to start with the updates

    Nfo's picture

    I have followed this manual that you have provided me: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/5.0-...

    I have managed to do the update, but I had to make some change or correction in the commands. I leave it here explained:

    In section 2:

    after using the command:
    sudo -u git -H git fetch

    I had to enter this to apply the changes and be able to continue:
    git stash

    In section 3:
    I do not settle with the vim text editor, so I have replaced it with nano:

    sudo -u git -H nano config.yml

    In section 4:
    I also change the editor by nano in this line:
    sudo -u git -H nano Gemfile

    I have a problem with this line:
    sudo -u git -H bundle install --without development test postgres --deployment

    It asks for the password of the user git, I do not know if there will be any by default, but nobody knows it. Before this problem use the root account:


    sudo -u root -H bundle install --without development test postgres --deployment

    In section 6:
    The line mysql> has misled me in the commands, so I gave an error, I used them like this:

    mysql -u root -p
    GRANT LOCK TABLES ON `gitlabhq_production`. * TO 'gitlab' @ 'localhost';
    \ q


    This is a clarification in case someone else happens.

    Once all the steps have been done, I can enter and check that everything is ok:

    Thanx for your help.... goin to the next step

    Jeremy Davis's picture

    Good work on that. It sounds like you are getting there. Thanks for posting your notes as I'm sure it will be a help to others. Hopefully, I'll have the new GitLab appliance ready soon...

    Nfo's picture

    good morning, I just started trying to migrate the server to version 6.0. I have a problem with step 0:

    When executing this command:

    sudo -u git -H bundle exec rake gitlab: backup: create RAILS_ENV = production

    I get an error in all the wiki sections. In my case they are not used, but before I did not give any error ...

    at the end, gives an error "rake aborted"

    I also get an error when executing this command:

    sudo -u git -H bin / rails console production

    He tells me there is no bin / rails ... so I can not do any checking ...

    How can I fix these problems? thanks for your help!

     

    Nfo's picture

    the rake aborted error occurs because I do not have the /home/git/gitlab/public/uploads folder created on the system.

    So applying step 8 at the beginning I can make the copy ....

    when executing the following command:

    sudo -u git -H bundle install --without development test mysql --no-deployment

    It does not do anything, it stays there thinking and does not advance ... apparently the server is online, but it does not download or install or see any activity ....

    >

    need help

    Jeremy Davis's picture

    Note that step 8 is creating the directory:

    cd /home/git/gitlab
    sudo -u git -H mkdir -p public/uploads
    sudo chmod -R u+rwX  public/uploads
    

    Also you are using the instructions for when GitLab is backed by PostgreSQL (instead of MySQL). AFAIK you should have MySQL DB in your instance?! Please double check and correct me if I am wrong.

    Assuming I'm correct and you do have MySQL (and not PostgreSQL), you should be using the MySQL section of the instructions (not the PostgreSQL ones). I.e. in step 5 use these commands (i.e. in the '# MySQL' section):

    # MySQL
    
    # Run a bundle install without deployment to generate the new Gemfile
    sudo -u git -H bundle install --without development test postgres --no-deployment
    
    # Install libs (with deployment this time)
    sudo -u git -H bundle install --without development test postgres --deployment
    

    You then need to skip the '# PostgreSQL' section. The next commands you need to run start after where it says '# Both MySQL and PostgreSQL'.

    Jeremy Davis's picture

    If you can take a snapshot of the VM at this point, then I wouldn't worry about taking a GitLab backup. So long as you have some way of rolling back if things go bad, that's all you need.

    Re your uploads directory missing, it seems that you need to create that as part of the upgrade.

    Regarding the "command not found" error, that appears to be caused by a misunderstanding on your part. There are 3 main ways you can execute an executable file directly. If it's in your PATH then you can just use the command, e.g. 'turnkey-version'. Otherwise, you can use an absolute path, e.g. '/usr/bin/turnkey-version'. Or you can also use a relative path, e.g. 'cd /usr; bin/turnkey-version'. Note that a relative path relies on you already being in the right place.

    The 'which' command will show you the full path to were an executable is.

    # which turnkey-version
    /usr/bin/turnkey-version
    

    The system PATH is all the file locations that your system will look for executables when you try to run a command (each one separated with a colon). See '/usr/bin' is in there:

    # echo $PATH
    /root/bin:/root/bin:/usr/lib/git-core:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

    As the command 'bin/rails' is a relative path, you will need to cd to the correct directory for that to work. Looking at the instructions, it looks like you need to be in '/home/git/gitlab' first. So try this:

    cd /home/git/gitlab
    sudo -u git -H bin/rails console production
    

    It should work then. If not, then things are really cooked!

    Jeremy Davis's picture

    I think that there might be some bug in the website. I "replied" to each of your separate posts, yet they both got posted on the end, and in reverse order?!.

    FWIW, this one (by me) was in reply to your "5.1.0 to 6.0 problems" post . And this one (from me) was a reply to your "rake aborted!" post from you... Hope it all makes sense...

    Nfo's picture

    ok, i try all tips and post result

     

    Thank you very much for your help

    Nfo's picture

    The first thank you for your help and the second is to apologize because I did not notice the steps, since you were right, my gitlab uses mysql.

    I have problems with step 5:
    When executing the command:

    sudo -u git -H bundle install --without development test postgres --no-deployment

    You spend almost an hour looking for info in step:
    Fetching source index from https://rubygems.org/

    and then I get the error:

    Unfortunately, a fatal error has occurred. Please see the Bundler
    troubleshooting documentation at http://bit.ly/bundler-issues. Thanks!
    /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/index.rb:95:in `block (3 levels) in unmet_dependency_names ': undefined method` name' for ["ruby- ajp ","> = 0.2.0 "]: Array (NoMethodError)

    I leave you a capture:



    and also a command log:

     


    root@gitlab git/gitlab# sudo -u git -H bundle install --without development test postgres --no-deployment
    Fetching https://github.com/gitlabhq/raphael-rails.git
    remote: Enumerating objects: 46, done.
    remote: Total 46 (delta 0), reused 0 (delta 0), pack-reused 46
    Unpacking objects: 100% (46/46), done.
    Fetching https://github.com/ctran/annotate_models.git
    remote: Enumerating objects: 16, done.
    remote: Counting objects: 100% (16/16), done.
    remote: Compressing objects: 100% (13/13), done.
    remote: Total 4792 (delta 6), reused 8 (delta 3), pack-reused 4776
    Receiving objects: 100% (4792/4792), 1.22 MiB | 976 KiB/s, done.
    Resolving deltas: 100% (2330/2330), done.
    Fetching gem metadata from https://rubygems.org/..
    Fetching source index from https://rubygems.org/
    Unfortunately, a fatal error has occurred. Please see the Bundler
    troubleshooting documentation at http://bit.ly/bundler-issues. Thanks!
    /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/index.rb:95:in `block (3 levels) in unmet_dependency_names': undefined method `name' for ["ruby-ajp", ">= 0.2.0"]:Array (NoMethodError)
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/index.rb:95:in `map'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/index.rb:95:in `block (2 levels) in unmet_dependency_names'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/index.rb:94:in `map'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/index.rb:94:in `block in unmet_dependency_names'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/index.rb:93:in `map'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/index.rb:93:in `unmet_dependency_names'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/source/rubygems.rb:240:in `remote_specs'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/source/rubygems.rb:163:in `fetch_specs'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/source/rubygems.rb:67:in `specs'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/definition.rb:192:in `block (2 levels) in index'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/definition.rb:189:in `each'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/definition.rb:189:in `block in index'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/index.rb:9:in `build'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/definition.rb:185:in `index'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/definition.rb:179:in `resolve'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/definition.rb:114:in `specs'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/definition.rb:109:in `resolve_remotely!'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/installer.rb:83:in `run'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/installer.rb:14:in `install'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/cli.rb:247:in `install'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/vendor/thor/task.rb:27:in `run'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/vendor/thor/invocation.rb:120:in `invoke_task'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/vendor/thor.rb:344:in `dispatch'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/vendor/thor/base.rb:434:in `start'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/bin/bundle:20:in `block in <top (required)>'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/friendly_errors.rb:3:in `with_friendly_errors'
            from /var/lib/gems/1.9.1/gems/bundler-1.3.5/bin/bundle:20:in `<top (required)>'
            from /usr/local/bin/bundle:23:in `load'
            from /usr/local/bin/bundle:23:in `<main>'
    root@gitlab git/gitlab#



    On the other hand and aprobechando your generosity, in step 6 refers to check the configuration of version 5 in version 6 ... Where can I look at the configuration that my server has now ??

    Thank you

    Nfo's picture

    I continue with the tests and I have completed the process, but the server failed me to do the tests and I get error 502 bad gateway

    I'll comment on the changes, to see if this gives us the error:

    In step 5:
    Given that there is something that does not work when installing with this command:
    sudo -u git -H bundle install --without development test postgres --no-deployment

    I have tried with this that I have seen on the internet:

    sudo -u root -H bundle install --full-index

    I have to do it with root, because with git it asks for a password and I do not know what it is, nor how to continue ... install everything without problems.

    In step 6:
    I copy the orginal to adapt it to the new format. I managed to get the service started without problems
    I think the problem is here

    In step 10:
    at the time of doing the checks I get this error:

    rake aborted!
    (<unknown>): did not find expected key while parsing a block mapping at line 10 column 1



    Logically the service does not work

    Let's see if you can light me a little ... since I'm almost almost on the point of achieving the goal.

    Thank you

     

    Jeremy Davis's picture

    I got a little tied up yesterday... (FWIW, I was working on our new GitLab appliance - which will be much easier to maintain...)

    Regarding the bundler issue that you've hit, my guess is that the version of Ruby, Rails and/or bundler needs updating. In my experience, generally the GitLab upgrade docs are pretty good at noting when particular versions need updating, although perhaps they missed something? According to the requirements doc for GitLab v6.0, you'll need Ruby v1.9.3. Although I also note, that in the install instructions, it demonstrates installing Ruby 2.0.0. Please note, that I DO NOT suggest that you follow that install doc blindly, but it may give you some hints on what you need to do.

    From what I can see, it looks like in the v13.0 GitLab appliance, Ruby is installed from the Debian repos. From what I can gather, the default Ruby was 1.8, but 1.9.3 was also available apparently. However, the info you've posted appears to suggest that you may have Ruby 1.9.1 installed?! Unfortunately I can't be 100% sure on any of that stuff as I'm struggling to find an authoritative source for info relevant to Debian Wheezy (what TurnKey v13.x was based on).

    So first up, refresh your apt package cache. That should all work smoothly and there is no need to post the output. However, if it shows any warnings or errors, please feel free to post them.

    apt-get update

    Then run the following commands and post back with the output (copy/pasting the text is preferable, but a screenshot will do if it's easier).

    which ruby
    ruby -v
    apt-cache policy ruby*
    

    I'm almost certain that something is the wrong version. Armed with the above info info, hopefully I will be able to guide you on the required next step. If you already have a source install of ruby, then updating that is likely the best path. Otherwise, it may be easier to install Ruby 1.9.3 from the Debian repos (if it is indeed available).

    Re your second post, the "502 bad gateway" error is expected behaviour when GitLab itself won't start. It basically means that the front end server (Nginx IIRC), can't connect to the backend server (GitLab in this case).

    Also you mention that "sudo -u git ..." asks for password?! It appears that you are logging in as root. By default a password should not be set for the 'git' user account and AFAIK if you are logged in as root, then you should not need a password to use sudo with a "normal" user account anyway. I'm not 100% certain, but I think that the following command should remove the user password and allow you to sudo without one:

    passwd -l git

    Re your comments about steps 6 onwards. If a particular step fails, then it's best to not trying to continue. You really need to resolve whatever is causing the issue in the problematic step (i.e. step 5 from what I can gather), before moving on.

    Containing past an error, will in a best case scenario just not work; in a worst case scenario, may actually break other things and cause further future issues. TBH, I think making things any worse is unlikely, but is possible...

    So we need to focus on working out why step 5 is failing and fix that issue before trying any further steps. As I note above, I'm almost certain that there is an incorrect version of something. When you're trying stuff though, be careful as it's likely that installing a version of any of the components which is too new, will also cause you issues!

    Jeremy Davis's picture

    Also, the link to the bundler troubleshooting guide (noted in one of the error messages you reported) appears to have changed (I got a 404). It appears that it has been moved to here.

    Nfo's picture

    I already have the git updated to version 6.0.2!
     


    First of all thank you for your time and help!

    Step to detail how I have managed to do:

    Before doing anything, I did step 8, since the public / uploads folder does not exist on my server:

    cd / home / git / gitlab

    sudo -u git -H mkdir -p public / uploads
    sudo chmod -R u + rwX public / uploads


    I also skip the step of backups, since I'm working with virtual machine backups. If anything happens, I start the previous one and that's it.

    Step 0:

    my database is called like this: gitlab_production

    Login via web to look at the name here:
    https://gitv2:12321/mysql/index.cgi

    I assign the database correctly:
    use gitlab_production;


    Step 2:
    After executing this command:

    sudo -u git -H git fetch

    I have to run this to continue and apply the changes

    git stash

    Step 5:
    I edit the files with nano, since I saw I do not like it:
    sudo -u git -H nano Gemfile

    I apply the changes:
    git stash

    here reset the password of the git user so he does not ask for it anymore:
    passwd -l git

    Since I find it impossible to do the installation with this command:
    sudo -u git -H bundle install --without development test postgres --no-deployment

    I use this one that I found online:
    sudo -u git -H bundle install --full-index

    Step 6:
    To avoid modifying the original files, I rename, it facilitates me to put the data in the file, to be blank.

    cd /home/git/gitlab/config /
    mv gitlab.yml gitlab.yml.old
    mv unicorn.rb unicorn.rb.old

    sudo nano /home/git/gitlab/config/gitlab.yml
    sudo nano /home/git/gitlab/config/unicorn.rb


    These changes are operational .... On the other hand, the gitlab.yml file gave me error in a space. I could check it thanks to a web page that I also found looking for google. This page analyzes the code and shows you where the error is, with this I solve my problems:

    http://yaml-online-parser.appspot.com/

    Paste the code in the first box and return the errors in the one on the left. This has been very useful in this step, since I suspected that in the files something was wrong.

    Once all the steps are finished, the server works without problems.


    In the final checks I get this notice in violet, but everything works fine.

    Sorry for my english... google translate help me...!

    Nfo's picture

    Hi, I've already started trying the update to 7.14. I get to step 7

    This step does not work for me:
    sudo -u git -H bundle install --without development test postgres -deployment

    I get the following error:



    I try with:
    sudo -u git -H bundle install --full-index
    bundle update gollum-lib

    Same problem.....

    Tomorrow I will continue with the investigation. Thank you

    Jeremy Davis's picture

    I'm impressed with how far you've got so far, with relatively few issues. So well done!

    Unfortunately, I can't help much with this particular issue. I've just had a quick google, but nothing of any use came up for me unfortunately. As a hint, if you copy paste the exact error into google and see what comes up (in case you weren't aware, selecting the text in PuTTY should also copy it to your system clipboard, then just paste it into your web search). If you don't get enough good results, start removing specific parts, such as full file paths and explicit version numbers, keep trimming until you get some useful results.

    Thanks for your kind words, and your English is fine! :) Good luck.

    Nfo's picture

    Good morning again:


     
    I have already managed to upload to version 7.14, but something is missing or I do not know what happens, because when updating a project with tortoisegit, it gives an error when writing. I do not know if it will have to do with the new features that have been implemented or something that I have skipped or I still have to do ..... Step to explain the modifications that I had to do this tutorial: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/6.x-...

    Step 4:
    I can not install nodejs because I can not find it, so this line does not work:
    sudo apt-get install nodejs

    I have used this one:
    curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -sudo apt-get install -y nodejs

    Step 7:

    If we do nothing, at the time of installation, the error that I posted in my previous post comes out:

    Downloading gollum-lib-4.0.2 revealed dependencies not in the API or the lockfile (rouge (~> 1.7.4)).
    Either installing with `--full-index` or running` bundle update gollum-lib` should fix the problem.


    To solve this problem that prevents gollum installation, you have to modify the Gemfile.lock file:

    sudo -u git -H nano Gemfile.lock

    You have to look for rouge in the file and change the versions to 1.7.4, in my Gemfile.lock it appears 2 times:

    gollum-lib (4.0.2)
          rouge (~> 1.9)     

    rest-client (1.8.0)
        rouge (1.9.1)

    We keep the changes and continue without problems.

    In step 8:

    You have to change this line in the gitlab file:

    cd /etc/nginx/sites-available
    sudo nano gitlab


    listen [::]: 80 ipv6only=on default_server;

    If I do not make this modification, I get this error when starting the nginx service:

    Restarting nginx: nginx: [emerge] bind () to [::]: 80 failed (98: Address already in use)

    In this section, I do not understand what to do:
    A new location / uploads / section has been added that needs to have the same content as the existing location @gitlab section.

    With this the server works. But if I do step 12, when I delete the gitlab user, the database stops working and the service does not start



    How can I solve the two problems I have:
    - I can not upload projects to gitlab.
    - finish step 12 without exploiting the gitlab server

    As always, thank you very much
    Jeremy Davis's picture

    Regarding the Nginx error "Restarting nginx: nginx: [emerge] bind () to [::]: 80 failed (98: Address already in use)", my guess is that there is an errant Nginx instance still running (not being controlled by the service). To check whether that is the case, stop the Nginx service and check whether anything is listening on port 80:

    netstat -tlnp | grep "80 "

    If Nginx is not running and you get any response from that, manually kill the process which is running. E.g. On a LAMP server I currently have running:

    tcp6       0      0 :::80                   :::*                    LISTEN      2959/apache2

    In my case it is an "apache2" process which is using port 80 and it's PID (process ID) is 2959. You can kill a process (with a PID of 2959) like this (adjust the PID as needed):

    kill -9 2959

    Another way would be to just reboot the server.

    Regarding your DB issues (within step 12), my guess is that you haven't completed the final steps (right at the bottom) regarding reconfiguring GitLab to use the new DB username ('git' instead of 'gitlab') and password. Note the error message "Access denied for user 'gitlab'..." shows that your GitLab instance is still trying to connect to the DB as the 'gitlab" user. I suggest that you double check and if need be, re-run the final steps:

    # Update database configuration details
    # See config/database.yml.mysql for latest recommended configuration details
    #   Remove the reaping_frequency setting line if it exists (removed in GitLab 6.8)
    #   Set production -> pool: 10 (updated in GitLab 5.3)
    #   Set production -> username: git
    #   Set production -> password: the password your replaced $password with earlier
    sudo -u git -H editor /home/git/gitlab/config/database.yml
    
    Nfo's picture

    I had not modified the password in the database.yml file. I had left the long number and had not changed my password. With this it is operative with all the changes!

    Jeremy Davis's picture

    Excellent. :)

    Nfo's picture

    This is error when i upload any file to gitlab:

    ------------------------------------------

    git.exe push --progress "origin" master:master

    Enumerating objects: 3, done.
    Counting objects: 100% (3/3), done.
    Delta compression using up to 4 threads
    Compressing objects: 100% (2/2), done.
    Writing objects: 100% (3/3), 6.52 KiB | 2.17 MiB/s, done.
    Total 3 (delta 0), reused 0 (delta 0)
    remote: /home/git/gitlab-shell/lib/gitlab_net.rb:16:in `check_access': undefined method `join' for #<Enumerator:0x00000002595f18> (NoMethodError)
    remote:     from /home/git/gitlab-shell/lib/gitlab_access.rb:23:in `exec'
    remote:     from hooks/pre-receive:13:in `<main>'
    To http://172.22.0.99/jc/test222.git
    ! [remote rejected] master -> master (pre-receive hook declined)
    error: failed to push some refs to 'http://172.22.0.99/jc/test222.git'


    git did not exit cleanly (exit code 1) (1750 ms @ 04/03/2019 15:50:17)

    --------------------------------------------------------------------------

    Jeremy Davis's picture

    I note that step 2 is updating Ruby, but according to this bug report it appears that the issue is caused by an older version of Ruby.

    IIRC by default, your server would have been using the system version of Ruby which is definitely too old. However, even if you installed an updated version of Ruby, it is possible that GitLab is still trying to use the old version of Ruby which your appliance shipped with.

    As noted in the upgrade notes, a quick way to check the version of ruby that your system is providing is to run:

    ruby -v

    If that is anything other than 2.1.6, then see where it is coming from:

    which ruby

    If that returns "/usr/bin/ruby", then you could try uninstalling the system Ruby package:

    apt-get remove ruby

    Let me know how all that goes.

    Nfo's picture

    I have tried to dismantle ruby. Restarted the system and I tried again. I have the same error. I can not publish modifications with the tortoisegit.

    I attached a screenshot of the commands that you have passed me. Ruby are uninstalled, but when restarting I keep having the same version.

     


    In the admin module of the web I also appear

    Thanx for help!

    Jeremy Davis's picture

    You certainly seem to have the correct version of Ruby (and GitLab Shell for that matter too). Both GitLab and the commandline appear to be showing the desired Ruby version.

    Regardless, all my googling only brings up issues with Ruby (and/or Gems) to produce that exact error. So I'm not really sure what is going on. Unfortunately due to my lack of Ruby and/or Rails knowledge, I'm running out of ideas on what to suggest next.

    The only other thing that I could suggest is that you double check for any Ruby related packages (in case they are named something else):

    apt-cache policy ruby* | grep -v "(none)" | grep -B 1 Installed

    That should show all the packages which are installed that start with 'ruby' in the name. Any that are installed will have a version number next to where is says "Installed:". Most that aren't installed shouldn't be shown, but there may be a few that still show, unless they have a version number next to "Installed:" they can be ignored. FWIW, as an example, here's what I have installed on my desktop (Debian Stretch so not directly relevant, but to show you want I mean):

    $ apt-cache policy ruby* | grep -v "(none)" | grep -B 1 Installed
    
    ruby2.1:
      Installed: 2.1.5-2+deb8u3
    --
    ruby2.3:
      Installed: 2.3.3-1+deb9u4
    --
    ruby-net-telnet:
      Installed: 0.1.1-2
    --
    ruby-json:
      Installed: 2.0.1+dfsg-3
    --
    ruby-minitest:
      Installed: 5.9.0-1
    --
    ruby:
      Installed: 1:2.3.3
    --
    ruby-power-assert:
      Installed: 0.3.0-1
    --
    rubygems-integration:
      Installed: 1.11
    --
    ruby-test-unit:
      Installed: 3.1.7-2
    --
    ruby-debian:
      Installed: 0.3.9+b6
    --
    ruby-did-you-mean:
      Installed: 1.0.0-2
    

    So in my case, I have "ruby" installed, but also additional ruby packages, namely "ruby2.1" (from Debian Jessie) & ruby2.3. So in your case, these packages should also be removed.

    If it still doesn't work, then actually, it's probably worth consulting the actual GitLab log to see if you can find full details about the error. That may note something specific which may head us in the right direction? TBH, I forget where the logs are by default. Either in /var/log somewhere, or otherwise somewhere within /home/git/gitlab. If you want to try searching, then these commands might help:

    # to search /home/git for files or directories which include "log" in the name:
    find /home/git -type d -name "*log*"
    # to search /home/git for files which include "log" in the name:
    find /home/git -type f -name "*log*"
    # to search /var/log for files and directories which include "git" in the name:
    find /var/log -name "*git*"
    
    Nfo's picture

    this server is testing to go up the versions little by little .... if I update the server with the error that we have of publication to version 8.0, do you think it will solve? I can try to do anything, since it is not a server in production .....

    Thanks for your time ;)

     

    I have this:

    root@Gitupdate ~# apt-cache policy ruby* | grep -v "(none)" | grep -B 1 Installed
    ruby1.9.1-dev:
      Installed: 1.9.3.194-8.1+deb7u8
    --
    grub-pc:
      Installed: 1.99-27+deb7u3
    --
    grub-pc-bin:
      Installed: 1.99-27+deb7u3
    --
    ruby-dev:
      Installed: 1:1.9.3
    --
    libruby1.9.1:
      Installed: 1.9.3.194-8.1+deb7u8
    --
    grub-common:
      Installed: 1.99-27+deb7u3
    --
    grub2-common:
      Installed: 1.99-27+deb7u3
    --
    ruby1.9.1:
      Installed: 1.9.3.194-8.1+deb7u8

    ------------------------------------------------------

     

     

    Jeremy Davis's picture

    I see below you have continued on. TBH, that was a pretty good idea, because my guess is that this issue would either continue to be a problem (and not really affect the upgrade path) or would resolve itself... So good work on that.

    Although FWIW it looks like you do have Ruby v1.9 installed, which should be safe to remove. Do that like this:

    apt-get remove ruby1.9.1 libruby1.9.1 ruby-dev ruby1.9.1-dev
    

    There may still be some left over bits that are no longer needed. The blow command will ensure they're all cleaned up. Although have a look at the packages that it plans to remove before allowing that to complete. If it gives you anymore than just a couple of packages that look ruby and/or development related, please feel free to cancel out of that for now and just post the list of packages here so we make sure you don't inadvertantly remove anything important.

    apt-get autoremove
    Nfo's picture

    Hello again!

    Doing tests I tried to update to version 8 to see if I had the same problem or if I succeeded.

    I have managed to update it to the first one and without errors .... !!! OMG

    The update works well, it has given me some error that had already happened to me, so on the fly I have been correcting them ....

    And finally I tried to upload and download files and do them without problems. It is definitely a problem of version 7.14 with ruby ​​.... !!

    As everything was very fast and I did not expect to be successful the first time, I did not have time to elaborate a step by step .....

    As I have to make a good manual for the updates and take them successfully, as soon as I have it I post it for you to record ...

    On the other hand, I have problems to login with the administrator. It does not let me login, I get the following error:

    If I close the browser and enter again, I can enter all sites, but the first time does not leave me ....

    Thank you very much ... for your time, your help and your post ... :)

    Jeremy Davis's picture

    Look at you go! :) Great job.

    I'm more than happy to assist. :)

    Re the issue of not being able to log in. I'm only guessing, but I reckon it's probably to do with a CSRF token.

    To check that out, I suggest that you restart GitLab (or even reboot the server) so everything is clean and fresh. Then open a fresh browser window (to ensure that there are no cookies on your end causing problems, or web browser plugins that are interfering, I would recommend using a "private"/"incognito" window). If you get that same issue, then there is something wrong and it's probably worth checking the logs. If you don't have that issue, then you should be all good.

    Nfo's picture

    if I enter with another browser or delete the cache I have the same problem ....

    It is already fixed, I just entered as administrator, I went to the users and I have renamed administrator to administrador. Already fixed, it works without problems.

     

    Thanks!!

     

    Nfo's picture

    If I make the change with what I have said in the previous post, then I can not install or do anything, since I give permission error .... this is secondary, I leave it to commentary mode in case someone ventures to perform change, know that the server does not work

    Nfo's picture

    Here we have a brief review of the modifications and problems found in this update.

    This tutorial is a mixture of these two:
    - https://github.com/turnkeylinux/tracker/wiki/Upgrade-GitLab-in-v14.0-and...
    - https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/7.14...

    Step 3:

    After this command:
    sudo -u git -H git checkout - db / schema.rb
    I have to execute this one:
    git stash
    Apply the changes and I can continue.

    Step 5:
    The same thing happens to me, I have to execute it after:
    cd gitlab-git-http-server
    git stash

    Step 7:

    I have the gollum error again, so you have to edit the gemfile.lock file:

    sudo -u git -H nano Gemfile.lock

    We search (rouge (~> 1.X.X)). And modify it to (rouge (~> 1.7.4)). It is in two places:

    gollum-lib (4.0.2)
          github-markup (~> 1.3.1)
          gollum-grit_adapter (~> 0.1,> = 0.1.1)
          nokogiri (~> 1.6.4)
          rouge (~> 1.9)

    rest-client (1.8.0)
          http-cookie (> = 1.0.2, <2.0)
          mime-types (> = 1.16, <3.0)
          netrc (~> 0.7)
        rinku (1.7.3)
        rotp (1.6.1)
        rouge (1.9.1)

    Step 9:
    You have to edit the file /etc/nginx/sites-available/gitlab to avoid the error:
    listen [::]: 80 default_server

    for this I execute the following commands:
    cd /etc/nginx/sites-available
    sudo nano gitlab

    I add a # just in front of the line listen [::]: 80 default_server
    ## listen [::]: 80 default_server

    With this already updates without problems. Tomorrow I start uploading to version 8.1

    My question is, until what version do I have to arrive?
    If I understand you correctly, I have to get to 9.3, then you can export the database and we can install version 15.2 and import the data from the backup.

    Do you reference omnibus ... is it the same platform that I'm using? or is it another distribution or different system?

    Thank you

    Nfo's picture

    Good morning, I'm still fighting with the gitlab ....
    I upload from version 8.0.5 to 8.1.4 without problems. Everything works correctly ... I can publish and download projects without problems

    I upload from version 8.1.4 to 8.2.6. I do the whole process without problems, following this guide: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.1-...

    In step 7:

    I do not know what to do with the section nginx-configuration . Here it only compares the files, but I do not know if it is necessary to edit, add or copy ... and I think that this is my problem ...

    I finish the whole process and everything gives me ok, I can enter via web, but when it comes to downloading or uploading a project with tortoisegit it gives me the following error ......


    -------------------------------

    fatal: unable to access 'http://......../': the requested URL returned error: 502

    Git did not exit cleanly (exit code 128)

    ----------------------------------
    According to I read it is a proxy problem, but I do not know where to change it or where to look ...

    thanks for your help
    Jeremy Davis's picture

    Yes, I agree that it sounds like a proxy problem. And almost certainly related to your Nginx config.

    So the instructions that you note (from the GitLab docs) show how to do a git diff of the Nginx config between v8.1 and v8.2. You should then apply those same changes to your existing Nginx config. I.e. if there are lines removed, then remove them from your config. If there are lines added, you should add them to your config.

    If you need a more explicit hand with that, perhaps you could post the output of the git diff command and I can attempt to explain it for you.

    Nfo's picture

    sorry but I had not posted the update to version 8.1.4 yet:

    It's the easiest update I've done for the moment, since I have not had any problems or anything strange ....

    In step 3:
    after the command:
    sudo -u git -H git checkout - db / schema.rb


    I had to execute this to be able to apply the changes:
    git stash

    Step 7:
    I apply these steps for ease when pasting:
    cd / home / git / gitlab / config
    mv gitlab.yml gitlab.8.0.5
    sudo nano gitlab.yml


    And after completing everything, I have version 8.1.4 operative.

    Nfo's picture

    already fixed, all the problem I had was in the proxy settings, since I had not done anything ... and logically it did not work.

    Step 7:
    This is where I had the problem when uploading or downloading files with tortoisegit.

    In the end the command to compare did not see anything clear, so I understood that it has to be removed and that it has to be added, but it seems very impractical.

    I have solved it like this:
    On this website I can see the gitlab.yml.example file of version 8.2. Checking I have seen the button to search for files. I searched for the gitlab-ssl and the gitlab for version 8.2, downloaded them and compared them with the files I have on the server. To compare what I have done with the notepad ++, since it is faster and easier to locate the changes ... I execute the following commands to rename the file of version 8.1 and I create a new one in white, I copy the file already corrected and to follow. ....

    cd /etc/nginx/sites-available
    mv gitlab-ssl gitlab-ssl.8.1.4
    sudo nano gitlab-ssl

    mv gitlab gitlab.8.1.4
    sudo nano gitlab


    Step 9:

    at the time of checking the status of everything tells me that a folder is missing, I run the following command to create it and do the check. Everything ok!

    sudo -u git mkdir -m 750 /home/git/gitlab/shared/lfs-objects

     

    Can you clarify a doubt? I have to get to version 9.3 no ?? at what point does it change from mysql to postgresql?

    thank you very much for your help

    Nfo's picture

    I'm doing tests to upload to version 8.3.10.

    I follow this tutorial:
    https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.2-...
    but I have problems with step 8:

    When executing the command
    redis-cli info | grep redis_version
    He tells me that I do not have a server

    If I continue with the tutorial to do the installation, at the time of loading the service with all the default parameters or changing the port to 7000 (I did it by going testing things) I can not load the service, it gives me an error in the configuration :

     

    On the other hand I can do the process of uploading and downloading projects without problems, but the redis service is not updated.

     

    Thanx !!

    Jeremy Davis's picture

    It sounds like there is some issue with Redis. I'm not certain, but my guess is that there is something in the config which is not compatible with the version of Redis you have. Either you have config too new for the version of Redis you have, or the other way round.

    Out of interest, do you know/recall how Redis is installed? Did you install it, or is it using the version of Redis that it shipped with the original appliance? (I know I could probably read back through your notes and/or the documentation, but I'm a bit pressed for time).

    Regardless, you'll most likely need to update it. FWIW install instructions can be found here. You may wish/need to remove the current Redis though. If it's installed via a Debian package, then it's a case of "apt-get remove PACKAGE_NAME" (where PACKAGE_NAME is the name of the redis pacakge, probably "redis-server" or perhaps just "redis"? - Sorry I can't be more precise).

    If it's installed from source (similar to the instructions I linked to above), then you should be able to just remove the files, where ever they are. Although all that really needs to be removed is the executable (which you should be able to find via "which redis" or perhaps "which redis-server" or something similar...

    Sorry my instruction feel a little vague and rushed, but as I say, I'm a bit pressed for time ATM and I don't recall all the steps that you have already gone through...

    Nfo's picture

    I found the error

    The problem with the update of redis 2.8 is that a command is missing from the list that appears on the web:

    # Build Redis
    cd / home / git / gitlab
    wget http://download.redis.io/releases/redis-2.8.23.tar.gz
    tar xzf redis-2.8.23.tar.gz
    cd redis-2.8.23
    make
    sudo make install

    with this it installs without problems! so problem solved.

    Jeremy Davis's picture

    This is actually a response to an earlier question you had re when you should switch to PostgreSQL DB backend. The short answer is when you migrate data to the Omnibus install (as opposed to the source install you currently have). It also covers your question re the version that you are aiming for (before you do the switch to the Omnibus package).

    My thinking was that once you get GitLab to v9.3.0, then you can migrate your data from the existing server, straight to a v15.2 GitLab server (which will include a newer version of GitLab by default, but will allow easy downgrade to v9.3.0 and much easier upgrade going forwards). IMO, that process would be best served by creating a new separate server, and migrate the data from the current server, to a new one. If creating a new server is not going to be an option for you, please let me know so I can adjust my advice.

    Unfortunately, I'm still not done with the new GitLab release and it's not yet publicly available, which is not ideal. However, I am still hopeful that it will be done this week, so may still be an option. If you are ready before I have finalised the new release, then all is not lost. You could migrate data to TurnKey v15.0 Core with GitLab installed from Omnibus package (as opposed to a source install like you currently have). I won't detail that right now, but I will note some relevant info (although not explicitly related to that) and if required, I am happy to provide some additional guidance if that's required.

    It's possibly worth noting, that there are other options. You could upgrade the OS itself and install the GitLab Omnibus package in your current server, but I suspect that it will be cleaner and less chance of major issues if you migrate data to a new (newer OS) server. Otherwise you would need to have GitLab installed twice on the single server and I suspect that could get messy... I thought that may get confusing and although I know the source updates are a bit time consuming and a pain, it seemed to me probably the best path. But really the choice is yours.

    Assuming that you wish to follow my suggestion, there are a few docs around that explain how to migrate from GitLab source install to Omnibus package (and migrating from MySQL to PostgreSQL is a component of that). Here are the first docs you'll want to read through:

    As noted in this post (different GitLab version, but still applies) if you migrate data from old server to new (i.e. 2 separate servers), you'll need to connect from the new server, to your current server's MySQL DB. That will require you to make the MySQL DB accept remote connections (by default that is disabled for security purposes). Again there are other ways, but IMO that will give you the cleanest result.

    I also found a quite detailed write up of following the process on a single server. That documents migration of v8.10.13 (and then bringing it up to v10.6.4). After a closer look, it appears that actually that may be a possibility in your case too?! I hadn't realised, but GitLab provide packages all the way up to v11.6.9 for Debian Wheezy (the basis of TurnKey v13.x) and they already provide a v8.2.6 and/or v8.3.10. So actually, if you wished, you could do that right now if you wanted!?! (Sorry I didn't notice that earlier).

    I hope my rambling and somewhat changing my suggestions within this single post doesn't confuse you. It's just that new info has come to light as I've been writing and because I'm in a bit of a rush, I'm not going back to edit what I've already written...

    Anyway, I hope that makes some sort of sense... Please post back if you need any clarification.

    Nfo's picture

    ok, according to what you mention, I also think that the best option is to go up to 9.3, make a backup of the data. Install the new version and recover the data in the latest available version.

    Thank you very much for your clarifications and for your time;)

    Nfo's picture

    I've been doing tests with this update all day, I get to do it without problems but for some reason the sidekiq cracks. When I do a check at the end to see that everything went well with the following command:

    sudo -u git -H bundle exec rake gitlab: check RAILS_ENV = production SANITIZE = true

    It shows me this error:

    Checking Sidekiq ...
    Running? ... do not
       Try fixing it:
       sudo -u git -H RAILS_ENV = production bin / background_jobs start
       For more information see:
       doc / install / installation.md in section "Install Init Script"
       see log / sidekiq.log for possible errors
       Please fix the error above and check the checks.

    I try the command and I still have the same problem.

    In this post they have the same problem and they give the solution, since it seems that there is an update of the GLIBC version, but I do not know how to do it ... to see if you can think of something ....

    Thank you

     

    Jeremy Davis's picture

    glibc is a fundamental core component of the Operating System (C is a low level programming language which many of the foundation OS components are written in - glibc is the common library for C on Debian/TurnKey).

    In theory you could install a newer version from source alongside the existing version, but in practice I would not advise you do that. I suspect that it would cause as many (or more) problems than it would solve.

    So I would recommend one of 3 options:

    1. Migrate your data to the Omnibus install now (via installing Omnibus package of same version of GitLab you now have, on your existing server, then migrate everything to this new install)
    2. Do an "in-place" Debian upgrade (i.e. upgrade the base OS to the next version (you currently have Debian 7/Wheezy base, so upgrade to Debian 8/Jessie)
    3. Ignore the errors for now and hope that they go away later (once you have migrated data to newer OS - which I think is likely)
    Nfo's picture

    ok, thanks for your support, support and wisdom:

    I'm going to try option 2 and if we do not go to 3

    I'm already telling you the progress!

    Nfo's picture

    I tried to upload the version of linux to 8 and I still have the problem, I have also uploaded to the 8.6 version making tests and I keep dragging the error of the sidekiq

    After being entangled and investigating I have realized where the error is, but I do not know how to correct it.

    The problem lies in the version of Redis that I install. When I do the tests and follow the manual, version 2.8.23 is installed. The problem is that if I check the sidekiq.log located in /home/git/gitlab/log I can see that it does not detect the version I have (2.8.23) if it does not detect a version 2.4.14.

    In the log I can see this:
    -------------------------------------------------- -
    2019-03-18T13: 18: 38.130Z 3140 TID-oxun5obzk INFO: Booting Sidekiq 4.0.1 with redis options {: url => "unix: /var/run/redis/redis.sock",: namespace => "resque : gitlab "}
    2019-03-18T13: 18: 38.138Z 3140 TID-oxun5obzk INFO: Cron Jobs - add job with name: stuck_ci_builds_worker
    2019-03-18T13: 18: 40.747Z 3140 TID-oxun5obzk INFO: Running in ruby ​​2.1.6p336 (2015-04-13 revision 50298) [x86_64-linux]
    2019-03-18T13: 18: 40.747Z 3140 TID-oxun5obzk INFO: See LICENSE and the LGPL-3.0 for licensing details.
    2019-03-18T13: 18: 40.747Z 3140 TID-oxun5obzk INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org
    You are using Redis v2.4.14, Sidekiq requires Redis v2.8.0 or greater
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/cli.rb:72:in `block in run '
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq.rb:84:in `block in redis'
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:64:in `block (2 levels) in with '
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:in `handle_interrupt '
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:in `block in with '
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `handle_interrupt '
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `with '
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq.rb:81:in `redis'
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/cli.rb:68:in `run '
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-4.0.1/bin/sidekiq:13:in `<top (required)> '
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/bin/sidekiq:23:in `load '
    /home/git/gitlab/vendor/bundle/ruby/2.1.0/bin/sidekiq:23:in `<top (required)> '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `load '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `kernel_load '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:28:in `run '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/lib/bundler/cli.rb:463:in `exec '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/lib/bundler/cli.rb:27:in `dispatch '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/lib/bundler/cli.rb:18:in `start '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/exe/bundle:30:in `block in <top (required)> '
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'
    /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.17.3/exe/bundle:22:in `<top (required)> '
    / usr / local / bin / bundle: 23: in `load '
    / usr / local / bin / bundle: 23: in `<main> '

    -------------------------------------------------- ------

    The problem I see is that I can upload version, everything works apparently, but I'm afraid that I explode, since it is a server that is in production .... I would like to know if this error I can fix it in some way.

    Thanks for your help, as always!

    Jeremy Davis's picture

    Apologies on delayed response (I actually wrote this yesterday, just negelected to actually post it... Oops).

    TurnKey GitLab v13.0 installed Redis from the Debian repos (see the plan here). So I'm not sure, but my guess is that even though you have installed a newer version, it is still seeing the Redis installed via package management first.

    You can remove it like this:

    apt-get remove redis-server

    Hopefully that will be enough, although if it still doesn't work (e.g. perhaps it will now complain it can't find redis at all) you may need to ensure that the newer redis is in the PATH (PATH is an environment variable that contains a list of paths to check for when trying to execute a binary).

    Nfo's picture

    I had already tried it and it does not work, if you wish it, the server keeps failing sidekiq ... although I just found the error and I have already solved it. I describe it to you in the next post! since it took me a week to trace it and solve it ....

     

    Thanx for your help!

    Nfo's picture

    I have used this manual for this step:
    https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.2-...

    But due to the multitude of problems, I have tended to do more steps that I now describe:

    Step 0:
    Before I start I need to create the lfs-objets folder because later I get an error and it tells me that it does not exist

    sudo -u git mkdir -m 750 /home/git/gitlab/shared/lfs-objects

    Step 3:
    I need to give permissions to the git user, since when executing any command it does not let me apply it for write permissions:

    sudo chown -R git .git /*

    Step 7:
    My gitlab does not have the /etc/default/gitlab file by default, so I create it according to this file: Gitlab

    cd /etc/default
    sudo nano gitlab


    Step 8:
    This step has given me many problems with sidekiq, because if I do what the manual says or it does not install it well, or it detects the old version:

    When installing, you must add these commands:
    cd redis-2.8.23
    make
    sudo apt-get install tcl
    sudo make install


    Avoid sidekiq errors, type You are using Redis v2.4.14, Sidekiq requires Redis v2.8.0 or greater.

    You have to edit the rescue.yml file and change the production line: unix: /var/run/redis/redis.sock
    by
    production: redis://localhost:6379

    cd /home/git/gitlab/config
    sudo nano resque.yml


    once finished, we must restart the redis service

    sudo service redis_6379 restart

    then we can check the version:
    redis-cli info | grep redis_version

    we will be in 2.8.23

    Once finished we can check the status of sidekiq:
    sudo -u git -H bundle exec rake gitlab: sidekiq: check RAILS_ENV = production

    Nfo's picture


    This update is done without problem with the steps that are here:
    https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.3-...

     

     

    Jeremy Davis's picture

    I have no doubt that this thread will be really useful for anyone else trying to do something similar, so thanks tons for posting back with progress notes.

    Regarding our updated GitLab release (v15.2). I have built it, but it's not yet published. It should be published by early next week I hope. So once you get GitLab to v9.3, I can assist you to migrate it to the new GitLab appliance. I'm actually really looking forward to that!

    And once you're on the new v15.2 GitLab appliance (which uses the GItLab Omnibus package), it should be super simple to maintain and keep up to date!

    Nfo's picture



    This update gives me an error when passing the tests at the end with the public/uploads folder, so before I start the process (which I repeat several times to make sure everything is ok before doing it in production):

    Step 0:
    Repair the Public/uploads folder:
    cd /home/git/gitlab/config
    rm -rf public
    sudo -u git -H mkdir -p public/uploads
    sudo chmod -R u + rwX public/uploads
    sudo chmod 0700 public/uploads


    Step 6:
    Here I omit the step of lipieza of gems, since in an attempt it has left me without styles in the portal .... I do not know 100% if it has been this. But as the idea is to migrate to another server and unsubscribe this .... I will not delete anything that is not strictly necessary.

    Nfo's picture

    this update following this manual: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.5-... is done without any kind of problems

    From the previous update I get this notice when performing the tests:
    Git version> = 2.7.3? ... no
       Try fixing it:
       Update your git to a version> = 2.7.3 from 1.7.10
       Please fix the error above and check the checks.

    As I read is for security issues, should upgrade to version 8 of linux .... but I have not yet been able to prove ... and for now it seems secondary.

    the next week I'll be a bit involved with other projects at work. So when I can return git ....

    Thanks for your help!

    Jeremy Davis's picture

    I must say that I think you are doing a fantastic job on this, especially for someone with limited GitLab and/or TurnKey experience. So good job!

    IIRC there are instructions on installing git from source in the GitLab docs, so that would be an option (i.e. remove the git-core Debian package and install git from source). Although, as you say, so long as everything works and you aren't getting any errors regarding that (i.e. only a warning) it's probably fine to leave it be for now. That will be automatically resolved once you move to the GitLab Omnibus package.

    The updated TurnKey GitLab appliance (v15.2 - with GitLab installed via Omnibus package) is built and should be published within the next few days.

    Nfo's picture

    Each time the updates are easier, I do not usually have problems or maybe it is already more updated and has less problems when updating.

    To make the update I followed this manual: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.6-...

    Everything works fine the first time, but I'm going to specify step 7:
    I downloaded the gitlab.yml file:
    I rename and edit the new one:


    cd /home/git/gitlab/config
    mv gitlab.yml gitlab.8.6.9
    sudo nano gitlab.yml


    I do the same procedure with nginx:
    Using the search engine of the website where you download the gitlab.yml file, you can search for the following files that need to be modified nginx files:

    cd /etc/nginx/sites-available
    mv gitlab-ssl gitlab-ssl.8.6.9
    sudo nano gitlab-ssl
    mv gitlab gitlab.8.6.9
    sudo nano gitlab


    Remember that you have to edit the gitlab file and comment on this line to avoid problems:
      ## listen [::]: 80 default_server;

    With all this the update is done without problems.
    When doing the tests it gives you the error commented in the previous post of the gitlab version. So far everything works fine, so I will not update the Linux version until it is strictly necessary.

    Nfo's picture

    I follow the tutorial: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.7-...

    This update is done without any problem, you just have to take into account two nuances that I will now detail:

    Step 6:
    In my case the database is not called gitlabhq_production, so I have to change the name. To locate it you must enter as administrator (https://gitlab:12321/mysql/index.cgi)

    GRANT REFERENCES ON `gitlab_production`. * TO 'git' @ 'localhost';

    In the last step, to exit the database you must remove the mysql> from the command:
    mysql> \ q

    With all this the update is done without problems

     

    Nfo's picture

    I follow the tutorial: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.8-...

    This update is done without any problem, just take into account the two modifications of the previous post to upload to version 8.8 in step 6

     

    Nfo's picture

    I follow the tutorial: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.9-...

    In step 6:
    Beware of the version of the database, the same thing happens to me as in the previous post.

    Step 7:
    Before you start you have to modify the Gemfile.lock file

    When executing the installation of mysql I get this error:


    Downloading omniauth-google-oauth2-0.4.1 revealed dependencies not in the API or the lockfile (omniauth-oauth2 (> = 1.3.1), jwt (~> 1.5.2)).
    Either installing with `--full-index` or running` bundle update omniauth-google-oauth2` should fix the problem
    .


    To fix this problem you have to edit the Gemfile.lock file:
    cd /home/git/gitlab
    sudo -u git -H nano Gemfile.lock


    You have to look for this part in the file, and change it to the one on the right in bold:
    omniauth-google-oauth2 (0.4.1)
         addressable (~> 2.3)
         jwt (~> 1.0) -----> jwt (~> 1.5.2)
         multi_json (~> 1.3)
         omniauth (> = 1.1.1)
         omniauth-oauth2 (~> 1.3.1) -----> omniauth-oauth2 (> = 1.3.1)

    Then we can continue with the command:
    sudo -u git -H bundle install --without postgres development test --deployment

    The rest of the steps are carried out without problems.

    Jeremy Davis's picture

    Sounds like you are working through the upgrade fairly smoothly now! Nice work and thanks again for posting your progress notes.

    Also, I've finally got the new GitLab appliance published. I haven't yet officially announced it (plan to do a blog post today hopefully) but it's already available for download via the appliance page.

    Once you get to v9.3, I'd really love to assist you to migrate all your GitLab data to the new appliance (which uses the Omnibus install). Once you're on the new appliance, updating to the latest will be a breeze!

    Nfo's picture

    I have followed this manual: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.10...

    This update has given me a bit of war. When I update something important, it is always accompanied by problems.

    Step to describe the errors and how to solve them:

    Step 4:
    After this command:
    sudo -u git -H git checkout - db /schema.rb
    I need to execute this other to be able to save the changes
    git stash

    Step 7:
    Here I have several problems, to update the ruby ​​version, bundler stops working, it has run out of gems .... So before you start with the installation of mysql, you have to do two things:

    cd /home/git/gitlab
    gem install bundler



    If I execute
    sudo -u git -H bundle install --without postgres development test --deployment

     

    I get this error:
    Downloading omniauth-google-oauth2-0.4.1 revealed dependencies not in the API or the lockfile (omniauth-oauth2 (> = 1.3.1), jwt (~> 1.5.2)).
    Either installing with `--full-index` or running` bundle update omniauth-google-oauth2` should fix the problem ..


    Before executing it, you must modify the Gemlock.file file
    sudo -u git -H nano Gemfile.lock

    omniauth-google-oauth2 (0.4.1)
          addressable (~> 2.3)
          jwt (~> 1.0) -> jwt (~> 1.5.2)
          multi_json (~> 1.3)
          omniauth (> = 1.1.1)
          omniauth-oauth2 (~> 1.3.1) -> omniauth-oauth2 (> = 1.3.1)

    And with this the update ends successfully!

    Nfo's picture

    This update has two nuances, but it is not problematic at all.

    Step 3:
    The ruby update can be omitted, since it is the same version that I have installed in the system.

    Step 4:
    You have to save the changes after the schema.rb, because it tells me that the Gemfile file is blocked.
    sudo -u git -H git checkout - db/schema.rb
    git stash


    The rest of the steps are done without problem.

    Nfo's picture

    I have followed the steps in this manual: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.12...

    Step 3:
    It is not necessary to update ruby, it is the same version

    The rest of the steps are correct

     

    Nfo's picture

    I have followed the steps in this manual: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.13...

    This update is also simple, since it does not present any serious problem:

    Step 4:
    When executing this command:
    sudo -u git -H git fetch --all

    I get this error:
    error: insufficient permission for adding an object to repository database .git / objects

    To fix it, before I start I give permission to the user git
    sudo chown -R git: staff .git

    sudo -u git -H git fetch --all


    and with this you can complete the entire tutorial without problems

    Nfo's picture

    I have followed the steps of this manual: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.14...

    This update has some problem with commands that I will describe and solve:

    Step 3:
    It is not necessary to update ruby, since we are in the same version

    Step 6:
    The command:
    sudo -u git -H bundle exec rake "gitlab: workhorse: install [/ home / git / gitlab-workhorse]" RAILS_ENV = production does not work.


    Something happens and it can not be executed.

    The way to solve it is to adapt this command to the one we used in previous versions. If I enter this web: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-15-stable/GITLAB_WORKHORS...

    I can check the version that has to have workhorse installed. In this case 1.2.1

    So, using these commands, we install it without problems:
    cd /home/git/gitlab-workhorse
    sudo -u git -H git fetch --all
    sudo -u git -H git checkout v1.2.1
    sudo -u git -H make


    The rest of the steps work without problems.

    Nfo's picture

    I follow this tutorial: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.15...

    Step 3:
    The echo command '1014ee699071aa2ddd501907d18cbe15399c997d ruby-2.3.3.tar.gz' | shasum -c - && tar xzf ruby-2.3.3.tar.gz is wrong, missing a space between c997d ruby. Once this error has been corrected, I can install it without problems.

    Step 5:
    In the mysql installation you have to do all these steps:

    Install the database packages
    sudo apt-get install -and mysql-server mysql-client libmysqlclient-dev

    # Ensure you have MySQL version 5.6 or later
    mysql --version

    # Secure your installation
    sudo mysql_secure_installation

    # Login to MySQL
    mysql -u root -p

    # Create a user for GitLab was already created
    CREATE USER 'git' @ 'localhost' IDENTIFIED BY '$ Password';  ##i update in another version

    #Insure you can use the InnoDB engine which is necessary to support long indexes
    SET storage_engine = INNODB;

    # If you have MySQL <5.7.7 and want to enable utf8mb4 character set support with your GitLab install, you must set the following NOW:
    GLOBAL SET innodb_file_per_table = 1, innodb_file_format = Barracuda, innodb_large_prefix = 1;

    # If you use MySQL with replication, or just have MySQL configured with binary logging, you need to run the following to allow the use of `TRIGGER`:
    SET GLOBAL log_bin_trust_function_creators = 1;

    # Create the GitLab production database -> in mi case, the name y gitlab_production
    CREATE DATABASE IF NOT EXISTS `gitlab_production` DEFAULT CHARACTER SET` utf8` COLLATE `utf8_general_ci`;

    # Eye my database is not called gitlabhq_production
    # Grant the GitLab user necessary permissions on the database
    GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES, REFERENCES, TRIGGER ON `gitlab_production`. * TO 'git' @ 'localhost';

    # Quit the database session
    \ q

    # Try connecting to the new database with the new user
    sudo -u git -H mysql -u git -p -D gitlab_production

    # Check for InnoDB File-Per-Table Tablespaces
    use gitlab_production;

    # Check your MySQL version is> = 5.5.3 (GitLab requires 5.5.14+)
    SHOW VARIABLES LIKE 'version';

    # Note where is your MySQL data dir for later:
    select @@ datadir;

    # Note if your MySQL server runs with innodb_file_per_table ON or OFF:
    SELECT @@ innodb_file_per_table;

    If SELECT @@ innodb_file_per_table returned 1 previously, your server is running correctly.

    Now, persist innodb_file_per_table and innodb_file_format in your my.cnf file. Ensure at this stage that your GitLab instance is indeed stopped.
    mysql -u root -p
    use gitlab_production;
    # Safety check: you should still have those values ​​set as follows:
    SELECT @@ innodb_file_per_table, @@ innodb_file_format;
    SELECT CONCAT ('ALTER TABLE `', TABLE_NAME, '` ENGINE = InnoDB;') AS 'Copy & run these SQL statements:' FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = "gitlabhq_production" AND TABLE_TYPE = "BASE TABLE";

    # Check for proper InnoDB File Format, Row Format, Large Prefix and tables conversion
    SET GLOBAL innodb_file_format = Barracuda, innodb_large_prefix = 1;

    # This should return the following:
    SELECT @@ character_set_database, @@ collation_database;

    # Convert tables not using ROW_FORMAT DYNAMIC:

    SELECT CONCAT ('ALTER TABLE `', TABLE_NAME, '` ROW_FORMAT = DYNAMIC;') AS 'Copy & run these SQL statements:' FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = "gitlab_production" AND TABLE_TYPE = "BASE TABLE" AND ROW_FORMAT! = "Dynamic";

    # Convert tables / columns not using utf8mb4 / utf8mb4_general_ci as encoding / collation:
    SET foreign_key_checks = 0;
    SELECT CONCAT ('ALTER TABLE `', TABLE_NAME, '' CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci; ') AS' Copy & run these SQL statements: 'FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA =" gitlab_production "AND TABLE_COLLATION! =" Utf8mb4_general_ci " AND TABLE_TYPE = "BASE TABLE";

    # Turn foreign key checks back on
    SET foreign_key_checks = 1;

    # You can now quit the database session
    \ q
    cd /home/git/gitlab

    sudo -u git -H editor config/database.yml


    # Modify the following fields:
    production:
      adapter: mysql2
      encoding: utf8mb4
      collation: utf8mb4_general_ci


    # Installations from source
    bundle exec rake add_limits_mysql RAILS_ENV = production

    Step 6:
    The installation I do manual, I check the version on this website:
    https://gitlab.com/gitlab-org/gitlab-ce/blob/8-16-stable/GITLAB_WORKHORS...

    and I update it as always:
    cd /home/git/gitlab-workhorse
    sudo -u git -H git fetch --all
    sudo -u git -H git checkout v1.3.0
    sudo -u git -H make


    and with these annotations the installation is completed without problem.

    Nfo's picture

    To update I follow this tutorial: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.16...

    Step 3:
    It can be omitted, since I already have this version intala

    Step 4:
    I have a version 6.15, so I do not need to update anything either

    Step 7:

    The command does not work:


    sudo -u git -H bundle exec rake "gitlab: workhorse: install [/ home / git / gitlab-workhorse]" RAILS_ENV = production

    So I do it manually, check version https://gitlab.com/gitlab-org/gitlab-ce/blob/8-17-stable/GITLAB_WORKHORS...

    cd /home/git/gitlab-workhorse
    sudo -u git -H git fetch --all
    sudo -u git -H git checkout v1.3.0
    sudo -u git -H make



    And with these corrections I upload a version without problems

    Nfo's picture

    I have followed this manual but with modifications: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/8.17...

    Step 0:
    Before starting I need to update the base version of linux to version 8. The problem is that I can not install libre2-dev since I have not found it for version 7.

    # Edit the /etc/apt/sources.list file again:

    nano /etc/apt/sources.list

    # and replace its content with the following lines:
    deb http://ftp.de.debian.org/debian/ jessie main contrib non-free
    deb-src http://ftp.de.debian.org/debian/ jessie main contrib non-free

    deb http://httpredir.debian.org/debian jessie-updates main contrib non-free
    deb-src http://httpredir.debian.org/debian jessie-updates main contrib non-free

    deb http://security.debian.org/ jessie / updates main contrib non-free
    deb-src http://security.debian.org/ jessie / updates main contrib non-free


    # Save file and execute:
    apt-get update
    apt-get upgrade


    I only do here, if I put the command
    apt-get dist-upgrade
    I miss the turnkey menus ... and at the moment I'm not interested.

    Step 3:
    The ruby ​​version you are requesting is already installed.

    Step 4:
    The version of node that I have is superior to the one that asks


    And with all these modifications I can do the update without problems. 
    three more updates and I'm ready for migration

     

    Jeremy Davis's picture

    Nice work! It looks like you are getting really close now!

    It's worth noting that unfortunately, the new GitLab appliance I've just finished has a bug... :( I'm working on releasing a fixed version (and including an option on first boot to choose the version to install), but I have also noted the workaround to the bug on the issue on our issue tracker.

    Nfo's picture

    In the end I have dared to make the jump from one version to another without going through all ....
    So I went from 9.0 to 9.3 with the following tutorial and some small extra annotations: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/9.2- to-9.3.md

    Step 3:
    No need to update, I have the version that asks

    Step 4:
    I just update yarm. Node I have it in version 6

    Step 6:
    This command does not work, and in no previous update it appears:
    sudo -u git -H git checkout -- locale
    so I ignore it ...

    Step 10:
    My database is called gitlab_production. You have to be careful with this to avoid making mistakes ....

    Step 11:
    Here is the key for everything to work well. In the update https://gitlab.com/gitlab-org/gitlab-ce/blob/11-8-stable/doc/update/9.0-... warns you that you have to modify the gitlab.yml file and modify the routes to:

      repositories:
        storages:
          default:
            path: /home/git/repositories
            gitaly_address: unix: /home/git/gitlab/tmp/sockets/private/gitaly.socket


    If you do not add the socket then the service does not load and the server does not work. You also have to be careful when editing the file, if you leave some blank space, when executing the error check in a line. If that happens to you, check with this web the file gitlab.yml: http://yaml-online-parser.appspot.com/ You will see that the error is in the lines that you have edited above.

    The rest of the commands work perfectly. I've noticed that every time the server has a higher ram consumption. So to do the tests and speed up I have uploaded the ram to 3 GB.

    You also have to enter the webmin and accept the change from linux version to debian 8 (I've seen it after the update).

    With this I am ready to move to the second phase, which will be the installation of the new version, export the data and migrate them to the new platform. If you can fix the error of 15.2 install the last one. If I do not put the 15.1 and we're updated.

    Nfo's picture

    Step 9:

    9. Update Gitaly

    cd /home/git/gitaly
    sudo -u git -H git fetch --all --tags
    sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION)
    sudo -u git -H make

    sudo mv env env.old

    sudo -u git cp config.toml.example config.toml

    Nfo's picture

    This part is finished, I put the webs that have helped me to get here:

    Check files: http://yaml-online-parser.appspot.com/

    Two Chinese pages that with the translator have helped me to repair some error:
    http://www.cnblogs.com/zy1234567/p/9940282.html
    https://www.cnblogs.com/zy1234567/p/9962508.html

    In this web it tells you the jumps for the versions to shorten the facilities:

    4.0 -> 5.0 -> 6.0 -> 7.14 -> 8.0 -> 8.2 -> 8.16 -> 8.17 -> 9.0 -> 9.3

    For lack of experience, the jumps from 8.2 to 8.16 have not been done, but I am sure that if I now feel and analyze the changes, I could do it without problems ... but I leave it for the next one.

    On the other hand I have a manual in word with all the steps. I also have all the files that are needed in txt to speed up the changes. When I have everything finished. I upload it to the internet and I post the link here to facilitate things the next.

     

    All the errors that I have solved have been with the help of san google ...
     

    Thank you very much for your support and your help. I certainly would not have gotten that far! You have been very kind and knowing that you are there for any question has helped me a lot.

    A thousand thanks again

    Jeremy Davis's picture

    Glad to hear that you're nearly there! And you are most welcome for the assistance. Other than a few pointers, most of the hard work has been done by you. So great work with that! :)

    Also thanks to you for posting all your progress notes along the way. As I said, I'm sure this thread (and your final document) will be of great assistance to others as well. So good work to you!

    So the next step will be to launch the new v15.2 appliance. Apologies that I haven't yet uploaded a fixed build that avoids the issue that I noted. Although, in your instance, the issue should not be a problem, as you'll need to remove the default setup to sync it with your current server.

    So in an effort to pre-emptively get you going in the right direction, download the v15.2 appliance, launch it and run through the firstboot scripts. Once you have done that, log in via SSH and run the following commands:

    gitlab-ctl cleanse
    apt remove gitlab-ce
    apt update
    apt install gitlab-ce=9.3.11-ce.0
    

    That should install a clean instance of GitLab v9.3.11 on your new server.

    Then the next step will be migrating your content to the new server. My previous post has some links which might be of benefit? Primarily these 2:

    As I also noted in that post:

    As noted in this post (different GitLab version, but still applies) if you migrate data from old server to new (i.e. 2 separate servers), you'll need to connect from the new server, to your current server's MySQL DB. That will require you to make the MySQL DB accept remote connections (by default that is disabled for security purposes). Again there are other ways, but IMO that will give you the cleanest result.

    Once you have transferred all your data to the new server, you can then use the Omnibus package to update to the latest version. That should be much quicker/easier than the manual update process that you've had to go through so far. According to the GitLab upgrade recommendations, you'll want to upgrade like this:

    apt install gitlab-ce=9.5.10-ce.0
    

    Then check everything is all good, assuming so, upgrade to v10:

    apt install gitlab-ce=10.8.7-ce.0
    

    Again check all is well. It may also be worth referring to the v10 release notes. Then once you have confirmed all is well, final step to upgrade to v11:

    apt install gitlab-ce=11.9.4-ce.0
    

    Again check all is well. Also see v11 release notes.

    Note too that v11.9.4 is the latest release at the time of writing. If there is a newer v11.x release, you can jump straight to that if you prefer. In theory, you could just upgrade to the latest version via:

    apt install gitlab-ce
    

    However, I didn't do that because I can't be sure when they plan to next do a major release. So stating a version means you won't accidentally jump straight to v12 version if GitLab release it between now and when you upgrade.

    FWIW you can check for all available GitLab versions using "apt policy gitlab-ce" (or "apt-cache policy gitlab-ce"). Personally I prefer to parse it with grep so only the relevant info is displayed. So I would check for an updated version like this:

    apt update
    apt-cache policy gitlab-ce | head -3
    

    On my current test server, that shows:

    gitlab-ce:
      Installed: 11.8.2-ce.0
      Candidate: 11.9.4-ce.0
    

    I.e. I currently have v11.8.2 and the latest is v11.9.4. Once GitLab release v12, then the above command will show the "Candidate" as "12.0.0-ce.0" (or whatever the latest version is at the time). FWIW the trailing zero is the package build; this allows them to release a new package of the same version of GitLab, e.g. if there is a problem with the package, but the version of GitLab stays the same.

    If you wish to follow what appears to be recommended practice of updating to the last release of a particular major version, before updating to the new major version, you can double check for the latest v11 release like this:

    apt-cache policy gitlab-ce | grep "     11\." | head -1
    

    FWIW use of the "\" character (e.g. before the "." above) within grep is referred to as "escaping". That is required because within grep's regex, a "." has a specific meaning (it will match any character!). So to ensure that it only matches a dot explicitly, you need to escape it.

    The above can be transferred to other version by changing the initial number (e.g. "12\.". Note too, that because in the version table, the currently installed version is preceded with "***", only "not installed" versions will show when the above is used (the installed version won't show). Use the first statement I suggested to check for the latest version (if you're on the latest the "Installed:" version will be the same and the "Candidate:" version).

    I hope that all helps... If you have further questions or problems, please don't hesitate to post back.

    Nfo's picture

    Good.

    Sorry I have not answered before, but I'm dealing with quality issues and I'm pretty messed up. I see that I have all the instructions. I hope that in two weeks I can free myself and attack the server again. I'm already commenting on the progress.


    Thank you

    Nfo's picture

    good morning!

    Today I have dedicated a little bit to gitlab, I already have the virtual machine created, I changed the base configuration to give it more ram and more hard drive, I upload it to 60gb and with the commands I can expand it without problems.

    Now I have this problem, after the base installation, the latest version of gitlab, I try to enter to see how it is and I can not. With the administrator I get 500 error ...

    As it is going to erase I do not care, but it is to take into account. If I enter the port: 12321 in services I do not see and mysql or PostgreSQL.

    Now let's go to the mess. I try to install postgre with the link that you have posted to me and it gives me privilege error:

     sudo gitlab-rake db: create db: migrate
    PG :: InsufficientPrivilege: ERROR: permission denied to create database
    : CREATE DATABASE "gitlabhq_production" ENCODING = 'unicode' LC_COLLATE = ''
    Could not create database for {"adapter" => "postgresql", "encoding" => "unicode", "collation" => nil, "database" => "gitlabhq_production", "pool" => 10, "username" "=>" gitlab "," password "=> nil," host "=>" / var / opt / gitlab / postgresql "," port "=> 5432," socket "=> nil," sslmode "=> nil , "sslcompression" => 0, "sslrootcert" => nil, "sslca" => nil, "load_balancing" => {"hosts" => []}, "prepared_statements" => false, "statements_limit" => 1000 , "fdw" => nil}
    rake aborted!
    ActiveRecord :: StatementInvalid: PG :: InsufficientPrivilege: ERROR: permission denied to create database
    : CREATE DATABASE "gitlabhq_production" ENCODING = 'unicode' LC_COLLATE = ''
    / opt / gitlab / embedded / bin / bundle: 23: in `load '
    / opt / gitlab / embedded / bin / bundle: 23: in `<main> '

    Caused by:
    PG :: InsufficientPrivilege: ERROR: permission denied to create database
    / opt / gitlab / embedded / bin / bundle: 23: in `load '
    / opt / gitlab / embedded / bin / bundle: 23: in `<main> '
    Tasks: TOP => db: create
    (See full trace by running task with --trace)



    all this gives me after deleting the latest version and installing the 9.3

    Nfo's picture

    I just realized that these commands are running on the old server .... sorry!
     
    Jeremy Davis's picture

    As I mentioned previously (perhaps you missed it, or perhaps misunderstood) the new v15.2 server has a bug which I need to fix. However, it shouldn't affect you as you'll need to downgrade GitLab to an older version (v9.3.11 - to match your old server). To repost the steps to downgrade GitLab version:

    gitlab-ctl cleanse
    apt remove gitlab-ce
    apt update
    apt install gitlab-ce=9.3.11-ce.0
    

    As for seeing different results in Webmin (e.g. not seeing PostgreSQL) that is because the GitLab Omnibus installer uses it's own version of PostgreSQL which isn't directly compatible with our usual tools. If you wish to use PostgreSQL, then you'll need to use the GitLab specific tool from the commandline. The command is 'gitlab-pgsql'. Other than how you launch it, it should be equivalent to the "normal" PostgreSQL commandline tool 'pgsql'. It's also worth noting that the PostgreSQL that is bundled with GitLab is configured to use a socket to connect. So no password is required, but any specific config and/or commands must be done via commandline.

    As for running commands, you may find the GitLab Omnibus "maintenence" doc page useful?!

    Nfo's picture


    I'm messing with the two servers. I understand that the migration to postgresql I have to do in the old server, not ?? that is, this manual is applied to my old server: https://docs.gitlab.com/ce/update/mysql_to_postgresql.html#source-instal...

     

    If so, I have a problem in this step:

    sudo -u git -H bundle exec rake db: create db:migrate RAILS_ENV=production

     

    when executing it I get the following error:

    sudo -u git -H bundle exec rake db:create db:migrate RAILS_ENV=production

    could not connect to server: No such file or directory

            Is the server running locally and accepting

            connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_recrd/connection_adapters/postgresql_adapter.rb:651:in `initialize'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `new'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `connect'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:242:in `initialize'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:44:in `new'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:44:in `postgresql_connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:438:in `new_connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:448:in `checkout_new_connect                                                                                                             ion'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:422:in `acquire_connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:349:in `block in checkout'

    /usr/local/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:348:in `checkout'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:263:in `block in connection'

    /usr/local/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:262:in `connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:571:in `retrieve_connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_handling.rb:113:in `retrieve_connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_handling.rb:87:in `connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/postgresql_database_tasks.rb:6:in `connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/postgresql_database_tasks.rb:15:in `create'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/database_tasks.rb:93:in `create'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/database_tasks.rb:107:in `block in create_current'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/database_tasks.rb:276:in `block in each_current_configuration'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/database_tasks.rb:275:in `each'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/database_tasks.rb:275:in `each_current_configuration'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/database_tasks.rb:106:in `create_current'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/railties/databases.rake:17:in `block (2 levels) in <top (required)>'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'

    /usr/local/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:100:in `top_level'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:176:in `standard_exception_handling'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/bin/rake:33:in `<top (required)>'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/bin/rake:23:in `load'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/bin/rake:23:in `<top (required)>'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli/exec.rb:74:in `load'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli/exec.rb:74:iN `kernel_load'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli/exec.rb:28:in `run'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli.rb:463:in `exec'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli.rb:27:in `dispatch'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli.rb:18:in `start'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/exe/bundle:30:in `block in <top (required)>'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'

    /usr/local/lib/ruby/gems/2.3.0/gems/bundler-2.0.1/exe/bundle:22:in `<top (required)>'

    /usr/local/bin/bundle:22:in `load'

    /usr/local/bin/bundle:22:in `<main>'

    Couldn't create database for {"adapter"=>"postgresql", "encoding"=>"unicode", "d                                                                                                             atabase"=>"gitlabhq_production", "pool"=>10}

    rake aborted!

    PG::ConnectionBad: could not connect to server: No such file or directory

            Is the server running locally and accepting

            connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `initialize'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `new'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `connect'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:242:in `initialize'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:44:in `new'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:44:in `postgresql_connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:438:in `new_connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:448:in `checkout_new_connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:422:in `acquire_connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:349:in `block in checkout'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:348:in `checkout'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:263:in `block in connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:262:in `connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:571:in `retrieve_connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_handling.rb:113:in `retrieve_connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_handling.rb:87:in `connection'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:916:in `initialize'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:823:in `new'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:823:in `up'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:801:in `migrate'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/database_tasks.rb:137:in `migrate'

    /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'

    /usr/local/bin/bundle:22:in `load'

    /usr/local/bin/bundle:22:in `<main>'

    Tasks: TOP => db:migrate

    (See full trace by running task with --trace)

    Jeremy Davis's picture

    On face value, it looks like PostgreSQL isn't running?! Although, perhaps it is, but not listening on a socket (or perhaps in the wrong place)?

    In the error message you've posted it says that GitLab is looking for a socket in the following location: /var/run/postgresql/.s.PGSQL.5432. So first thing I'd do is ensure that PostgreSQL is running. Something like:

    service postgresql status
    

    (Although perhaps the service is just called 'postgres' or something similar? You should be able to use Tab complete to find the right name).

    If it's not running, try starting it (as above but 'start' instead of 'status'). If it's refusing to start, then double check for error messages in the log (which is likely /var/log/postgresql or something similar - either do an 'ls' or use Tab complete again).

    Once you've confirmed that it is definitely running, then double check for the existence of the socket that GitLab is looking for. I.e.:

    ls -l /var/run/postgresql/.s.PGSQL.5432

    IIRC PostgreSQL should use a socket by default. But if the socket doesn't exist (and it's definitely running), then you may need to change the configuration of PostgreSQL so that it uses a Unix socket. I'm not 100% sure how to do that OTTOMH, but hopefully google will help you out there.

    Nfo's picture

    I'm rolling:

    What do I have to do exactly? when working with two servers I do not know in which of them I have to do it. I explain

    - In 15.2, I delete the gitlab and install version 9.3 (this I have clear)
    I have to do the rest of the steps in the new server ?? that is, install postgresql, create the database, transfer the mysql data from the old server .....

    I think I'm doing it wrong, since I'm trying to install the postgresql on the old server ... and I think I'm doing it wrong.

    The error that I gave you yesterday gave me because I did not have the postgresql installed on the old server.

    Clarify me where I have to do the steps. Thank you

    Jeremy Davis's picture

    My previous response was trying to assist you with the issues that you were hitting, but in retrospect I probably should have been trying to understand exactly what you were doing (and why). The problem is that I'm not completely familiar with the migration process myself, so I assumed that you were following some instructions that you had found.

    Anyway, I think that the best course of action would be to follow the GitLab docs for this scenario. They do note at the top of that page, that it is not thoroughly tested, but seems like the best option IMO.

    Strap yourself in, this is going to be a big one! :)


    Preparation

    The first important (albeit probably obvious) point to make is that this process will need to be done when no one (else) is writing anything to your GitLab server. Only data that is in the backup you are about to create will be included in the new server, so any data pushed during or after the backup occurs may/will be missing. If that's not currently possible, but you have time to work on this, it's probably still useful to do a test run with most of the data (then redo it later with all the data).

    Create a backup

    First step is to create a backup of your GitLab on your OLD server. Hopefully this part will be pretty straight forward.

    As noted in the relevant section, the backup will be saved to the backup_path directory, as set within the config file (should be /home/git/gitlab/config/gitlab.yml). TBH, I don't think it's set to by default, but here's an easy way to find out:

    grep backup_path /home/git/gitlab/config/gitlab.yml

    If that returns something, then that's the directory where the backup should end up. If it doesn't, try adding it to the "production" section of the config file (it needs to be valid yaml). So it should look something like this:

    production:
      adapter: mysql2
      encoding: utf8
      reconnect: false
      [other settings here...]
      backup_path: /home/git/gitlab/backups
    

    You'll also need to ensure that the path exists and is owned by the git user. To make sure:

    mkdir -p /home/git/gitlab/backups
    chown -R git:git /home/git/gitlab/backups
    

    Then you should be good to go, although you may need to restart GitLab to pick up the new settings. If for some reason you get an error after adding that, then you may need to consult the example config file. I forget where it is, but it should be in there somewhere and IIRC it's called 'gitlab.yml.example' or something similar. You should be able to find it like this:

    find /home/git/gitlab -type f -name '*gitlab.yml*'

    Note that will find your config file, but it will also find any files that include gitlab.yml in the filename. Hopefully that helps you find it. Once you've added a backup_path in a valid format, then you can move on to actually creating the backup.

    Just follow the instructions on the relevant GitLab docs page. As this is on your OLD server, you'll need to follow the "source install" instructions. Seeing as it's a production server, I would recommend that you use the "copy" backup strategy, but it may not matter if it's not under heavy usage.

    Transfer backup and config/secrets files to NEW server

    Note that you'll need to deal with your config files separately, as noted in the docs (as it's not included in the backup). You'll also need to convert your config file to the Omnibus format, but we'll worry about that later...

    Once you have your backup, then transfer it, and your config files to the new server. SCP is an easy option for that. You can do it either way (from the OLD server to the NEW one, or vice versa) but for arguments sake, I'll show copying from the OLD server to the new one:

    scp /home/git/gitlab/backups/BACKUP_FILENAME root@NEW_SERVER_IP:/root/
    scp /home/git/gitlab/config/gitlab.yml root@NEW_SERVER_IP:/root/
    scp /home/git/gitlab/config/secrets.yml root@NEW_SERVER_IP:/root/
    scp /home/git/gitlab/.secret root@NEW_SERVER_IP:/root/
    scp /home/git/gitlab/.gitlab_shell_secret root@NEW_SERVER_IP:/root/
    

    All 5 files should now be the /root directory (i.e. root's home dir) of your NEW server.

    Migrate DB

    The next step is to migrate the DB. There are a few different ways that can be done. One is to install MySQL on your NEW server, or alternatively install PostgreSQL on your old server. The former is probably the superior option of the 2, but they will both involve some mucking around. So I think the easiest way to go, is to enable remote connections to MySQL on your OLD server, then connect to it from your NEW server. That way you won't need to muck around installing and configuring too much additional software.

    Migrate DB - enable remote MySQL connections

    To do that, you'll want to allow remote MySQL connections (by default only local connections are allowed). Please check our DB docs for how to do that. Note that the "v15.0+" step of adding a new user should also work on older TurnKey/Debian servers, but it hasn't been tested. Let me know if you have any questions or hit any problems enabling remote connections.

    Once that is done and working, you shouldn't need to do anything further on the OLD server. We still need to migrate the DB, but even then, I don't recommend that you destroy it, until you are 110% sure that everything is good on your NEW server.

    So from here on, everything should be done on your NEW server.

    Migrate DB - test access to MySQL from NEW server

    First up, I recommend that you double check that you can connect to MySQL on the OLD server, from your NEW server, just to make sure that it works:

    apt update
    apt install mysql-client
    mysql -u USERNAME -pPASSWORD -h OLD_SERVER_IP
    

    (Swap out 'USERNAME', 'PASSWORD' and 'OLD_SERVER_IP' for the real values you're using)

    You should see something like this:

    Welcome to the MariaDB monitor.  Commands end with ; or \g.
    Your MariaDB connection id is 2
    Server version: 10.1.26-MariaDB-0+deb9u1 Debian 9.1
    
    Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
    
    Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
    
    MariaDB [(none)]>
    

    Assuming that works ok, you can close the client (Ctrl-C to exit).

    Migrate DB - install pgloader

    Next up is converting the MySQL DB to PostgreSQL (as included in the new GitLab appliance). As noted in the docs, you'll need pgloader v3.4.1+. Unfortunately, Debian Stretch (the basis of TurnKey v15.x) only has v3.3.2, so you'll need to install it from upstream. The docs show adding the repo, but seeing as you'll only need this once, IMO it's better to just install it by downloading the deb file and installing it that way. Do that like this:

    apt update
    cd /tmp
    wget http://apt.postgresql.org/pub/repos/apt/pool/main/p/pgloader/pgloader_3.5.2-3.pgdg90%2b1_amd64.deb
    apt install pgloader_3.5.2-3.pgdg90+1_amd64.deb
    

    I tested that on a v15.0 Core server that I had handy, so I'm confident that it should work fine on your TurnKey v15.2 GitLab server.

    Migrate DB - prepare to migrate data

    Once you have pgloader installed, you need to configure it to load the data from your OLD server, and convert to PostgreSQL on your NEW server. Assuming that remote MySQL connections are working correctly on your OLD server, all this can be done on your NEW server.

    The next step is actually covered in the GitLab docs, but some of the steps will require some tweaks, and some of the steps are unnecessary because of our default config. So I'll cover that whole section here.

    # stop GitLab
    gitlab-ctl stop
    # restart Postgres and Unicorn
    gitlab-ctl start unicorn
    gitlab-ctl start postgresql
    # ensure that the DB schema is all good - possibly not required, but let's do it anyway
    gitlab-rake db:create db:migrate
    # stop unicorn again
    gitlab-ctl stop unicorn
    

    In the next bit, you'll need to make sure that you set the values of the first 3 lines, replacing bits as noted in the comments.

    # create the pgloader commands.load file
    MYSQL_USER=YOUR_MYSQL_USER # replace 'YOUR_MYSQL_USER' with the relevant username
    MYSQL_PASS='YOUR_MYSQL_PASS' # replace 'YOUR_MYSQL_PASS' with the relevant password
    OLD_SERVER_IP=YOUR_OLD_SERVER_IP # replace 'YOUR_OLD_SERVER_IP' with the relevant IP address
    
    cat > /tmp/commands.load <<EOF
    LOAD DATABASE
         FROM mysql://$MYSQL_USER:$MYSQL_PASS@$OLD_SERVER_IP/gitlabhq_production
         INTO postgresql://gitlab-psql@unix://var/opt/gitlab/postgresql:/gitlabhq_production
    
     WITH include no drop, truncate, disable triggers, create no tables,
          create no indexes, preserve index names, no foreign keys,
          data only
    
     ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public'
    
     ;
    EOF
    

    Migrate DB - do the migration

    First ensure that everyone has access to the commands.load file, then to start the DB migration:

    chmod 777 /tmp/commands.load
    sudo -u gitlab-psql pgloader /tmp/commands.load
    

    As noted in the GitLab docs (step 3 of the pgloader instructions) you should see a summary table showing the import, ensure that there are no errors. As also noted, if you don't see output for more than 30 minutes, it’s possible pgloader encountered an error. If so, have a look at the troubleshooting steps. If you get stuck, please post back and I'll do my best to help out.

    Restore your your backup

    The next step is restoring your backup. First off, you'll need to consider migrating your secrets and config. If you want to start afresh, that can be skipped (the default config should be fine, and the secrets can be regenerated). If you want to migrate your secrets, this GitLab issue discusses how to do that.

    Then:

    # start GitLab again
    gitlab-ctl start
    # copy your backup into the backup dir & grant correct ownership
    cp /root/BACKUP_FILENAME /var/opt/gitlab/backups/
    chown git:git /var/opt/gitlab/backups/BACKUP_FILENAME
    

    Then follow the rest of the steps as noted in the GitLab restore docs. Hopefully that should all work smoothly and your GitLab should be fully functional.

    Update to latest GitLab

    Final step is to update to the latest GitLab. This should now be super easy! Just like this:

    apt update
    apt install gitlab-ce
    

    You should now be running the latest GitLab release on your new v15.2 GitLab server! In future you can update GitLab via the same step.

    Nfo's picture

    Thank you very much Jeremy !!

    Now I have all the steps clear, there is an error in them that I have already solved.
    I get to do the whole process, but there are things that fail. I will try to do more tests and after returning from my vacation (a week) I will tell you to help me. Something happens to me when it comes to restoring the backup to the new server, since it does the process but it ends up giving error ... Something happens to me with gitlab, since when I delete the new server to put 9.3.11, this one it never loads anymore ...

    Next week I tell you about the progress. Thanks for your time and help, it would have been very difficult to get here!
     

    Add new comment