Daniele Lolli (UncleDan)'s picture
Long story short. I made a little script to fix permissions for Moodle and it turns out that it is useful tu use it also for turnker-moodle. As it needs little adaptations to be included in tkl main (and also as it can be useful also for the installed machine if you mess up something), am I allowed to download and run it directly form a the repo? something like: git clone MYREPO -o /scripts chmod +x /scripts/MYSCRIPT /MYSCRIPT -p1 -p2 -p3
Forum: 
Jeremy Davis's picture

Hi Uncle Dan, sorry for such a slow response...

I'm a bit bogged down with the v19.x update. It should have been released AGES ago, but I'm still battling with a few gnarly issues (primarily the installer).

I'm guessing you've probably worked it out by now, but bottom line yes, what you're doing is fine. Ultimately it doesn't really matter where you put it, but personally - assuming that it's a script you want accessible from everywhere - I like cloning repos like that into /usr/local/src and then linking to the executable in /usr/local/bin - then it will be in your PATH (so you don't need to put the full path or ./ prefix to run it).

E.g. I'd do something like this:

cd /usr/local/src
git clone MYREPO
ln -s /usr/local/src/MYREPO/MYSCRIPT  /usr/local/bin/MYREPO/MYSCRIPT

Also if you run the 'chmod +x MYSCRIPT' in your git repo and commit the change, the executable bit will be retained. Files in a git repo will inherit their owner and user group at clone time, but the read/write/execute permissions are saved in the repo.

FYI just in case you've heard that you shouldn't touch files in /usr - that is indeed (generally) correct as the contents of /usr directories are (mostly) managed by the system (i.e. files from packages etc). However /usr/local is the exception. It can be user managed and that's where we put our custom scripts if they aren't packaged. It keeps them separate from system files but also keeps them all in the same spot. As I noted above in TurnKey /usr/local/bin is already in the PATH env var by default, so as long as the script is executable you can call the command from anywhere.

[update] - I just noticed that the last line of the command example above was incorrect/incomplete and have updated/fixed it now. FWIW the previous line would have created a link to the repo directory, not the actual script...

Daniele Lolli (UncleDan)'s picture

I know you're super busy but I'll try: no hurry :-) Trying to apply your suggestion but stuck on an error: it seems a curl error, but I don't use curl in my script

Module unique_id already enabled
Module security2 already enabled
Module evasive already enabled
fab-chroot build/root.patched --script /turnkey/fab/common/conf/php
updating /etc/php/8.2/apache2/php.ini
- memory_limit=128M
- post_max_size=16M
- upload_max_filesize=8M
- max_execution_time=30
- max_input_vars=1000
- opcache.interned_strings_buffer=8
- opcache.revalidate_freq=30
updating /etc/php/8.2/cli/php.ini
- memory_limit=128M
- post_max_size=16M
- upload_max_filesize=8M
- max_execution_time=30
- max_input_vars=1000
fab-chroot build/root.patched --script /turnkey/fab/common/conf/mysql
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 293k 100 293k 0 0 746k 0 --:--:-- --:--:-- --:--:-- 745k
/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3988 100 3988 0 0 21316 0 --:--:-- --:--:-- --:--:-- 21326
/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 14 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 404
make: *** [/usr/share/fab/product.mk:577: build/stamps/root.patched] Error 22


https://pastebin.com/raw/PsEjbTjk
Jeremy Davis's picture

The issue is in line 12 in /turnkey/fab/common/conf/mysql and appears to be caused by an upstream bug (the vulnerabilities.csv file disappeared).

As noted in that issue if you update the url to be the upstream dev's personal repo it should work again. I.e. change line 11 to use 'jmrenouard/MySQLTuner-perl' instead of 'major/MySQLTuner-perl'. The new line:

URL="https://raw.githubusercontent.com/jmrenouard/MySQLTuner-perl/master"

Having said that, as the name suggests its a script for performance tuning MySQL/MariaDB for your the specific server resources/use case. It's a "value add" and not necessary so unless you're actually using it you could simply remove (or comment out) the lines that do that (lines 11 - 17).

Although we should be noting a specific commit ID in that conf script and updating it each time we do a release. Then we could output a warning if the commit we're using at built time doesn't match the one that was last tested and confirmed working.

TBH I don't actually recall why we're installing from upstream regardless because there is a debian mysqltuner package. My guess it was because the 3 files we pull from the repo are orders of magnitude smaller than the Debian package? Regardless, we would probably be better off using that as our default install will never be updated. Or perhaps we should provide an install/update script instead of what we currently do? Or perhaps we should leave it out all together and document installing and using it somewhere? I suspect that most TurnKey users don't even realize that it exists - let alone that it's pre-installed...

Thoughts?

[update] as an additional note, regardless of the missing csv file the URL that the mysql conf script is using is abbreviated. That URL still works for me, but in discussions with my colleague he noted that all the downloads failed during his testing. I'm not sure why that is but I suspect that the URL currently in the mysql conf script is an old one and it redirects to the new (more specific) one for me and either I have a local cache and/or the redirection was broken during my colleagues testing. TL;DR we probably should be including the full/new URL as well as pointing to the upstream maintainers repo.

Daniele Lolli (UncleDan)'s picture

I understand that https://github.com/turnkeylinux/common/blob/18.x/conf/mysql#L12 is downloaded at some point during the building process, so I suppose I have to sed the old url with the new one at some point, but where? at the very beginning of the main? I will do some tries, but help would be appreciated to learn something new
Jeremy Davis's picture

As the actual issue is the broken URL in that common mysql conf script it will affect all apps that include mysql/mariadb - so it's best fixed in common because that will fix it everywhere; for all apps that include MySQL/MariaDB.

Beyond fixing it in common being the "best" fix - pragmatically it's the only way to fix the issue.

To explain why I say that: It is technically possible to fix it within the appliance code itself - but would require these steps:

  • copying the common mysql conf script into the appliance conf/ dir (rename it some thing like '00-mysql' to ensure that it runs first).
  • fix the url in the copied mysql conf script (note: only step required if you just fix it in common)
  • edit the appliance 'Makefile" and removing the inclusion of the common lamp makefile (the line 'include $(FAB_PATH)/common/mk/turnkey/lamp.mk'
  • then include/add all the specific parts of the common lamp.mk to your appliance 'Makefile' - to ensure that all the other relevant conf scripts and overlays from common are still applied - other than that buggy mysql conf script
  • As may be obvious from the above - the appliance 'Makefile' tells the build process which parts of common to include. For convenience there are common make files in common (*.mk files in common/mk) so overlays and conf scripts can be grouped together without needing to explicitly note each one in the app 'Makefile'.

    If you have a look at the lamp.mk file in common, you can trace which other make files/conf script/overlays are being included, I hope that makes sense as gives you some additional insight. If you want/need a little more info ask and I'll be back ASAP.


    update: - after posting, I re-read your post again and realized that there may be a vital bit of info you're missing: the "common" directory (inc that mysql conf script) is already on your TKLDev! Check out /turnkey/fab/common - so its not downloaded at build time; it's already there! :) Note that that is a git repo too. Although unlike appliances, it has branches for each major version (please just ignore 'master'). I.e. The current "default" branch is '18.x' and is used for all v18.x appliances. The common code for the next major version we're working on is in the 19.x-dev branch. Once we're ready to release v19.0 appliances, we'll rename the dev branch '19.x' and make that the default common branch. FWIW your current TKLDev will still continue to use the 18.x branch unless you manually change it - although to build v19.x apps it's recommended to use a v19.x TKLDev.

    FWIW during a new major version transition we initially start doing beta builds of v19.0 on a v18.x TKLDev. Once we have a local v19.0-beta ISO that installs (or we do a manual install) we use that for continued dev. The reason why things have been taking so long is because there have been a few fundamental issues - the biggest and the real "showstopper" is getting the installer working - because without an installer obviously an ISO isn't much use to most...

    Jeremy Davis's picture

    Good on you for having a go! :) As my grandfather used to say "nothing ventured, nothing gained". Hopefully my post above helps you understand how to address the issue!?
    Daniele Lolli (UncleDan)'s picture

    I vibe-coded a hotfix script to run on TKL dev before making the app, if it can be useful to someone: https://github.com/UncleDan/turnkeylinux-apps-test-18/blob/retest-moodle... Now I can build the old version, let's see if I can git myscript now.

    Add new comment