Paul's picture

Hello

Installed the canvas-lms and logged in successfully, looked at sis imports via a remote curl command. I do this regular on other systems so confident my curl and p/k are correct.

The sis imports recognised that there is a batch to process but just sits there and doesnt process any batches.

Tried the webadmin to kick the canvas_init start but get this:

Executing /etc/init.d/canvas_init start ..

 

`/var/www` is not writable.
Bundler will use `/tmp/bundler20230126-14497-j8wd3x14497' as your home directory temporarily.
Could not find command "bundle".

Any help on either stopping and binning the batch process or making sure it completes?

Paul.

Forum: 
Jeremy Davis's picture

FYI, we have a major version release of our Canvas appliance in the works. Hopefully it should be ready within the next week or so.

In the meantime, that doesn't sound good at all.

I am only guessing, but seeing that message, the first thing I'd think is that the canvas_init service is trying to run as the wrong user? Or the permissions on the Canvas dir (/var/www/canvas IIRC) aren't right?

I know we have users running the currently available Canvas instance, so I would expect it to "just work". Although to be clear, that doesn't necessarily rule out a bug (or bugs).

Can you please give me the output of the following:

ls -l /var/www/canvas

And

systemctl status canvas_init
Paul's picture

root@canvas ~# ls -l /var/www/canvas
total 1688
-rw-r--r--    1 root     www-data    6243 May 28  2021 CONTRIBUTING.md
-rw-r--r--    1 root     www-data     621 May 28  2021 COPYRIGHT
-rw-r--r--    1 root     www-data    4712 May 28  2021 Dockerfile
-rw-r--r--    1 root     www-data      65 May 28  2021 Dockerfile.githook
-rw-r--r--    1 root     www-data    3760 May 28  2021 Dockerfile.jenkins
-rw-r--r--    1 root     www-data    3387 May 28  2021 Dockerfile.jenkins-cache
-rw-r--r--    1 root     www-data     653 May 28  2021 Dockerfile.jenkins.final
-rw-r--r--    1 root     www-data     287 May 28  2021 Dockerfile.jenkins.karma-builder
-rw-r--r--    1 root     www-data     652 May 28  2021 Dockerfile.jenkins.karma-runner
-rw-r--r--    1 root     www-data     147 May 28  2021 Dockerfile.jenkins.linters-final
-rw-r--r--    1 root     www-data     644 May 28  2021 Dockerfile.jenkins.linters-runner
-rw-r--r--    1 root     www-data     139 May 28  2021 Dockerfile.jenkins.webpack-builder
-rw-r--r--    1 root     www-data     536 May 28  2021 Dockerfile.jenkins.webpack-cache
-rw-r--r--    1 root     www-data     353 May 28  2021 Dockerfile.jenkins.yarn-runner
-rw-r--r--    1 root     www-data     876 May 28  2021 Dockerfile.package-translations
-rw-r--r--    1 root     www-data    1291 May 28  2021 Gemfile
drwxr-xr-x    2 root     www-data    4096 May 28  2021 Gemfile.d
-rw-r--r--    1 www-data www-data   29754 May 28  2021 Gemfile.lock
-rw-r--r--    1 root     www-data   24991 May 28  2021 Jenkinsfile
-rw-r--r--    1 root     www-data    1470 May 28  2021 Jenkinsfile.cassandra
-rw-r--r--    1 root     www-data    4232 May 28  2021 Jenkinsfile.contract-tests
-rw-r--r--    1 root     www-data    1478 May 28  2021 Jenkinsfile.dive
-rw-r--r--    1 root     www-data    4311 May 28  2021 Jenkinsfile.docker-smoke
-rw-r--r--    1 root     www-data    3200 May 28  2021 Jenkinsfile.docker-sync
-rw-r--r--    1 root     www-data    1413 May 28  2021 Jenkinsfile.dynamodb
-rw-r--r--    1 root     www-data    4901 May 28  2021 Jenkinsfile.js
-rw-r--r--    1 root     www-data    1669 May 28  2021 Jenkinsfile.master-bouncer-check-all
-rw-r--r--    1 root     www-data    1749 May 28  2021 Jenkinsfile.package-translations
-rw-r--r--    1 root     www-data    4037 May 28  2021 Jenkinsfile.parallel_logs
-rwxr-xr-x    1 root     www-data    1476 May 28  2021 Jenkinsfile.postgres
-rw-r--r--    1 root     www-data    1564 May 28  2021 Jenkinsfile.redis
-rw-r--r--    1 root     www-data    1522 May 28  2021 Jenkinsfile.selenium-chrome
-rw-r--r--    1 root     www-data    9056 May 28  2021 Jenkinsfile.selenium.flakey_spec_catcher
-rw-r--r--    1 root     www-data    4652 May 28  2021 Jenkinsfile.selenium.performance.chrome
-rw-r--r--    1 root     www-data    1019 May 28  2021 Jenkinsfile.test-subbuild
-rw-r--r--    1 root     www-data    5869 May 28  2021 Jenkinsfile.vendored-gems
-rw-r--r--    1 root     www-data    2271 May 28  2021 Jenkinsfile.xbrowser
-rw-r--r--    1 root     www-data   34520 May 28  2021 LICENSE
-rw-r--r--    1 root     www-data     702 May 28  2021 README.md
-rw-r--r--    1 root     www-data     356 May 28  2021 Rakefile
drwxr-xr-x   14 root     www-data    4096 May 28  2021 app
-rw-r--r--    1 root     www-data    2016 May 28  2021 babel.config.js
drwxr-xr-x    2 root     www-data    4096 May 28  2021 bin
-rw-r--r--    1 root     www-data     244 May 28  2021 bower.json
drwxr-xr-x    6 root     www-data    4096 May 28  2021 build
-rw-r--r--    1 root     www-data    1676 May 28  2021 code_of_conduct.md
drwxr-xr-x    8 root     www-data    4096 Jan 24 16:53 config
-rw-r--r--    1 www-data www-data     267 May 28  2021 config.ru
drwxr-xr-x    3 root     www-data    4096 May 28  2021 db
drwxr-xr-x   10 root     www-data    4096 May 28  2021 doc
drwxr-xr-x   10 root     www-data    4096 May 28  2021 docker-compose
-rw-r--r--    1 root     www-data     233 May 28  2021 docker-compose.new-jenkins-contract-tests.yml
-rw-r--r--    1 root     www-data     215 May 28  2021 docker-compose.new-jenkins-flakey-spec-catcher.yml
-rw-r--r--    1 root     www-data     184 May 28  2021 docker-compose.new-jenkins-js.yml
-rw-r--r--    1 root     www-data     734 May 28  2021 docker-compose.new-jenkins-package-translations.yml
-rw-r--r--    1 root     www-data    1072 May 28  2021 docker-compose.new-jenkins-selenium.yml
-rw-r--r--    1 root     www-data     599 May 28  2021 docker-compose.new-jenkins.consumer.yml
-rw-r--r--    1 root     www-data     457 May 28  2021 docker-compose.new-jenkins.pulsar.yml
-rw-r--r--    1 root     www-data     327 May 28  2021 docker-compose.new-jenkins.vendored-gems.yml
-rw-r--r--    1 root     www-data    1268 May 28  2021 docker-compose.new-jenkins.yml
-rw-r--r--    1 root     www-data    1679 May 28  2021 docker-compose.spring.yml
-rw-r--r--    1 root     www-data     454 May 28  2021 docker-compose.yml
drwxr-xr-x    3 root     www-data    4096 May 28  2021 engines
drwxr-xr-x    3 root     www-data    4096 May 28  2021 frontend_build
drwxr-xr-x   59 root     www-data    4096 May 28  2021 gems
-rw-r--r--    1 root     www-data    3755 May 28  2021 gulpfile.js
drwxr-xr-x    2 root     www-data    4096 May 28  2021 hooks
-rw-r--r--    1 root     www-data     396 May 28  2021 issue_template.md
drwxr-xr-x    3 root     www-data    4096 May 28  2021 jest
-rw-r--r--    1 root     www-data    2086 May 28  2021 jest.config.js
-rw-r--r--    1 root     www-data    5257 May 28  2021 karma.conf.js
drwxr-xr-x   33 root     www-data   12288 May 28  2021 lib
drwxr-xr-x    2 www-data www-data    4096 Jan 27 00:00 log
drwxr-xr-x 2101 root     root       73728 May 28  2021 node_modules
-rw-r--r--    1 root     www-data   14457 May 28  2021 package.json
drwxr-xr-x   65 root     www-data    4096 May 28  2021 packages
drwxr-xr-x   13 root     www-data    4096 May 28  2021 public
drwxr-xr-x    3 root     www-data    4096 May 28  2021 script
drwxr-xr-x   31 root     www-data    4096 May 28  2021 spec
drwxr-xr-x    6 www-data www-data    4096 Jan 24 16:49 tmp
-rw-r--r--    1 root     www-data    4753 May 28  2021 ubuntu.development.Dockerfile
-rw-r--r--    1 root     www-data    3010 May 28  2021 ubuntu.production.Dockerfile
drwxr-xr-x    7 root     www-data    4096 May 28  2021 ui
drwxr-xr-x    3 root     root        4096 May 28  2021 vendor
-rw-r--r--    1 root     www-data     763 May 28  2021 webpack.config.js
-rw-r--r--    1 root     www-data    8543 May 28  2021 webpack.test.config.js
-rw-r--r--    1 root     www-data 1139536 May 28  2021 yarn.lock

Paul's picture

root@canvas ~# systemctl status canvas_init
* canvas_init.service - LSB: Start/stop Canvas background jobs
   Loaded: loaded (/etc/init.d/canvas_init; generated)
   Active: failed (Result: exit-code) since Fri 2023-01-27 16:10:25 UTC; 2min 51s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 493 ExecStart=/etc/init.d/canvas_init start (code=exited, status=15)

Jan 27 16:10:22 canvas systemd[1]: Starting LSB: Start/stop Canvas background jobs...
Jan 27 16:10:23 canvas su[493]: (to www-data) root on none
Jan 27 16:10:23 canvas su[493]: pam_unix(su:session): session opened for user www-data by (uid=0)
Jan 27 16:10:25 canvas canvas_init[493]: Could not find command "bundle".
Jan 27 16:10:25 canvas su[493]: pam_unix(su:session): session closed for user www-data
Jan 27 16:10:25 canvas systemd[1]: canvas_init.service: Control process exited, code=exited, status=15/n/a
Jan 27 16:10:25 canvas systemd[1]: canvas_init.service: Failed with result 'exit-code'.
Jan 27 16:10:25 canvas systemd[1]: Failed to start LSB: Start/stop Canvas background jobs.

Paul's picture

Forgot to add a thank you :)

Jeremy Davis's picture

So first up, it looks like the Canvas dir (/var/www/canvas) is owned by root, but the cavnas_init is running as www-data. I think that might be a factor?

I've had a quick look and I think I have a plan:

Log into your Canvas instance via SSH (or you can use Webshell - don't use the Webmin console!). First thing to do is stop the canvas_init service.

/etc/init.d/canvas_init stop

Then make sure that there are no stray canvas_init processes running:

kill -9 $(pgrep canvas_init)

(If that command gives an error, you can safely ignore it - it just means that there weren't any stray processes).

Now grant www-data ownership of /var/www/canvas:

chown -R www-data:www-data /var/www/canvas

Now tweak the init script. You can find it at /var/www/canvas/scripts/canvas_init. Line 16 should look like this:

  exec su $(stat -c %U $(dirname $(readlink -f $0))/../config/environment.rb) -c "/bin/bash $0 $@"

Edit that line so that it looks like this:

  exec su www-data -s /bin/bash -c "/bin/bash $0 $@"

(FYI - we're just hardcoding some of the stuff that it's trying to work out itself).

Start it up:

systemctl start canvas_init

And double check that it's running as it should:

systemctl status canvas_init

Look for the line that starts "Active: " (e.g. in your post above, it's: "Active: failed"). I'm pretty sure that if it's working, it should report "Active: active (running)". If yours reports anything other than that, please post back with the output of "systemctl status canvas_init".

Hopefully that will get you up and running.

Paul's picture

Hello, Thank you for this but no cigar.

I was off yesterday so just seen your reply.

First command gave an error:

root@canvas ~# /etc/init.d/canvas_init stop
Could not find command "bundle".

Jeremy Davis's picture

Just continue to the next step. Sorry, I should have been clearer. As a general rule it's a great idea to stop if/when you hit errors, especially when you're not completely sure of what you're doing. But in this case it doesn't matter.

Just continue to the next step. And if the next one errors, you can safely ignore that too (although I was clearer on that one).

Shouldn't be any errors after that though! (Fingers crossed).

Paul's picture

Thank you, i get this error now:

root@canvas .../canvas/script# nano canvas_init
root@canvas .../canvas/script# systemctl start canvas_init
Warning: The unit file, source configuration file or drop-ins of canvas_init.service changed on disk. Run 'systemctl daemon-reload' to reload units.
Job for canvas_init.service failed because the control process exited with error code.
See "systemctl status canvas_init.service" and "journalctl -xe" for details.

I gave the server a kick and tried the status command and this returned:

root@canvas ~# systemctl status canvas_init
* canvas_init.service - LSB: Start/stop Canvas background jobs
   Loaded: loaded (/etc/init.d/canvas_init; generated)
   Active: failed (Result: exit-code) since Tue 2023-01-31 11:05:51 UTC; 1min 8s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 493 ExecStart=/etc/init.d/canvas_init start (code=exited, status=15)

Jan 31 11:05:48 canvas systemd[1]: Starting LSB: Start/stop Canvas background jobs...
Jan 31 11:05:48 canvas su[493]: (to www-data) root on none
Jan 31 11:05:48 canvas su[493]: pam_unix(su:session): session opened for user www-data by (uid=0)
Jan 31 11:05:51 canvas canvas_init[493]: Could not find command "bundle".
Jan 31 11:05:51 canvas su[493]: pam_unix(su:session): session closed for user www-data
Jan 31 11:05:51 canvas systemd[1]: canvas_init.service: Control process exited, code=exited, status=15/n/a
Jan 31 11:05:51 canvas systemd[1]: canvas_init.service: Failed with result 'exit-code'.
Jan 31 11:05:51 canvas systemd[1]: Failed to start LSB: Start/stop Canvas background jobs.

Jeremy Davis's picture

Sorry for slow response... I got a bit tied up.

That sounds really broken. I had expected the 'Could not find command "bundle".' message to go away, once you were running as the right user.

I'm sure that it's possible to fix, but it sounds far from ready to use. I think that at this stage, my efforts are better spent trying to push the updated Canvas appliance out, rather than trying to debug this issue with a build that is quite old now.

I'm really sorry if that throws a spanner in your plans. But I genuinely think that's the best path for both of us. As for ETA of the new build, I can't promise anything, but the ISO should be ready within the next week and publicly available week after (fingers crossed). If you're desperate, I may be able to allow you access to the ISO before it's publicly released.

Paul's picture

No need for apologies at all :)

Im happy to bin the vm seeing as its poorly and time invested making it work would be better spent on the new build due soon.

Love to have access to the new build before release if that alright, i can then install and test.

Great product as well from you guys, massive well done, the lms, webadmin, ssh etc, great stuff.

Add new comment