TKL GitLab: Downloading archive is broken

Hi Royce,

Unless you enjoy mucking with GitLab, I suggest you consider dropping the TKLX GitLab VM and create a GitLab VM using TKLXCore + GitLab omnibus.

See my comment /comment/31033#comment-31033

Just a suggestion :-)

Each release GitLab has been adding more and more to its import / export feature. You may find that you can go from the old source-based version to the new omnibus version with a few export / import button presses (fingers crossed).

Regarding the database:

- In the newest GitLab some features are only supported on Postgresql (e.g. subgroups)

- Omnibus installer includes PostgreSQL, Redis, Sidekiq, etc so you only need to start with a TKLXCore VM (don't use the TKLX postgresql VM as the base)

- GitLab Omnibus lifecycle manages the database AND its interrlationships with appropriate support components (Redis, Sidekiq, etc). TKLX does NOT lifecycle anything on their VMs between major releases.

I would honestly recommend you switch from source-based GitLab to Omnibus-based GitLab sooner rather than later. GitLab is a complicated piece of software. No one but the original designers should touch it. Lifecycle managing it from source would be costly, error prone, and outside of GitLab support. If you use the omnibus you can report errors to GitLab which they will fix if it is serious enough. They won't necessarily address it if it is an issue with a source-based build.

If you look at GitLab pricing model you will notice something significant...they are a product company. They don't have a chargeable support tier like most open source initiatives. Because of this, the omnibus tier is the only way to go for a commercial deployment unless you have a reason to want to customise it.

NOTE: You will need to install GitLab Omnibus on TKLX V15.1. GitLab omnibus no longer supports the TKLX V14.x or earlier Debian.

CAVEAT: You will need to sort out a custom TKLBAM configuration.

Just one man's two cents worth :-)

Omnibus is the prefrerred management tool for GitLab, most of the documentation is written for those using it. Omnibus deals with a lot of challanges that I have been faced with while upgrading from source.

The only problem is that Omnibus only supports Postgres as the local DB service, my TKL started with MySQL and has been upgraded to MariaDB. We are also using MariaDB at work so this helps with adoption of GitLab to replace GitList.

I will consider your advice for the next TKL container that I setup.

Thank you.

First up, thanks for reporting your issues. Even if it is just something on your end, it's great to be aware of any potential issues. It may be something we've missed, although we test appliances, our testing isn't exhaustive and sometimes bugs sneak through.

Personally, I'm not aware of the issue that you noted, but it sounds like at least initially it was a GitLab bug?! If you have a bug ID, you should be able to check the source code (I'd start with the changelog) and work out when/where the issue was fixed so you'll at least know whether it should be fixed now or not. Judging from your post, my guess is that it's a combination of a bug (initially) and some missing or incorrect configuration, or perhaps a missing or incorrect version dependency.

Unfortunately, I don't have anything more specific to add to your experience, other than to ensure that you read through each single step of the upgrade from the GitLab docs. Whilst it is often possible to skip versions when updating, it's worth at least reading the upgrade docs of each individual upgrade as sometimes there are significant changes in one particular version which might be missed if you jump too many versions at once.

Re the issue you encountered with the v15.1 appliance, thanks heaps for reporting that. I've opened a bug relating to that and will look into it ASAP.

Thanks to Tim for bringing up the Omnibus installer for discussion. It actually ties in with an internal discussion we've been having of late. We're considering moving the TurnKey GitLab appliance from a "source" install to an "omnibus" install and would really appreciate some feedback/insight/thoughts from you guys (and anyone else who is interested). I've opened a feature request for that.

Personally, I think that the source install is pretty nice, but it is a bit of a headache to maintain, both from our perspective and for end users such as yourselves. I have no experience with the Omnibus install and it sort of goes against the way we usually do things here at TurnKey (it installs lots of stuff that we'd generally prefer to get via official Debian archives). OTOH if it adds value and improves the user experience (bonus if it also reduces our maintenance overhead) then it's a "no brainer"!

Love to hear any thoughts! :)

I found that the git command is called locally in the server to build the archive. I wanted to know how it is being ran so I created a wrapper script to dump the env and command.

I found that the archive fomat type is set to null, that sounds wrong but it may not be cause of the problem.

My debugging with the wrapper stopped when I found that the git program is also needing to read STDIN, possibly a pipe. My BASH script can't do that. To wrap this could require the bash script to open a persistant handle, that is not something BASH does easily. I may re-write the script in Perl, this would allow for asyncronous reads from STDIN and writes to STDOUT. If there is already such a tool, please let me know.

I did get 11.4 running, it too creates zero byte archive files. 

I will come back to this in a few weeks, busy with other things. The 11.6 release is going to be very attractive as it has a serverless implamentation. We have been using serverless for a while now and it is a key component to our hybrid cloud. 

I will post new findings when I have them.