George3430's picture

Hi Guys,

I've installed a copy of InvNinj 17.2 iso as a VM on proxmox I'm trying to add users to the installation. It saves the config ... and it says that it sent out an invitation to the new user to choose and confirm their password. However the invitation email never arrives !? It seems to be unique to the Turnkey image, as I have an exact mirror (in terms of InvNinj)  on an instance of Ubuntu 22.04 installed which works fine. Below is the email part of the .env file, obviously all addresses and pwd's are fictitious 

******

MAIL_MAILER="smtp"
MAIL_HOST="mail.themailhost.address.somewhere"
MAIL_PORT="465"
MAIL_USERNAME="theboss@mail.themailhost.address.somewhere"
MAIL_PASSWORD="letmeinprettypls"
MAIL_ENCRYPTION="ssl"
MAIL_FROM_ADDRESS="theboss@mail.themailhost.address.somewhere
MAIL_FROM_NAME="Dev Invoice Ninja"

********

Same config on both ... the only major difference is that the Ubuntu InvNinj VM is hosted on a TrueNAS Core box and I did the whole build from scratch ... Where as the TKL version is an image hosted on a proxmox server..

As both the configs are identical I suspect it might be an access rights or firewall issue on the TKL VM Image ..

Is there an error log somewhere that I can check?.. pardon the ignorance  ... as I mentioned before I'm a networking guy playing with server stuff

Any ideas?

Regards

George V

 

                                                                                                                             

Forum: 
Jeremy Davis's picture

Before I give you what I hope is the solution to your issue, it's perhaps worth sharing how we pre-configure it. We include Postfix (local MTA - mail transfer agent - i.e .a basic mail server backend) in all our servers and configure the web app (Invoice Ninja in this case) to use send mail via the local postfix service.

Postfix is quite capable of sending mail directly, but because of spammers, the chance of your emails being delivered without being flagged as spam is low. In fact mail sent directly from Postfix (without any of the accompanying anti-spam config - such as DNS entries etc) will often just be silently dropped - so it won't even land in your spam folder! However, the local Postfix can be configured to use a SMTP relay. Confconsole provides "Mail Relay" config.

The advantage of using the locally installed postfix via a third party mail relay (rather than the app using the remote SMTP relay directly) is that you will then also get system emails. The system emails will alert you to things like failing security updates and other things that are well worth being aware of on a production server.

Having said that, there's nothing stopping you from doing it the way you are trying to. You can still set up the system emails separately if you want (if you want the system emails too).

Regardless, to explicitly share the issue I think you're hitting:

When changes are made in the .env file, Laravel based web apps generally need to be "optimized" to pick up the config changes. FYI Laravel is a PHP web app framework that Invoice Ninja and many other modern PHP apps are built on. I won't go into detail, but many (most? all?) settings from the .env file (including 'MAIL_MAILER' email config AFAIK) are only read when the app is "optimized" (the command builds a local cache of PHP classes). So many (most? all?) .env file changes won't make any immediate difference - even if you restart the webserver.

I suspect that that is why it "just works" on your self installed version, but not on your TurnKey server. You would have run "artisan optimize" as part of the install process - and almost certainly after you had configured the .env file as desired. But not when you manually changed the .env file on TurnKey, my guess is that you didn't know you needed to run that command. Normally docs will suggest running the command like this:

cd /var/www/WEBAPP
php artisan optimize

However it's best to run that command as the webserver user ('www-data'). Otherwise, you'll likely need to change permissions after running the command. To make life easier for you, we provide a CLI tool to run the artisan command from anywhere (so you don't need to cd to the dir) and runs it as the webuser. Run that like this:

turnkey-artisan COMMAND

So to run "artisan optimize" is as simple as:

turnkey-artisan optimize

Hopefully that resolves your issue and provides some explanation. If you continue to have issues, please check for the Laravel log file. It should be /var/www/invoiceninja/storage/logs/laravel.log. Please feel free to copy paste from that here.

There are also some email troubleshooting steps noted in their docs.

Hopefully that gets you headed in the right direction. If not, please post back and I'll try some more...

George3430's picture

Hi Jeremy,

After I installed the appliance on proxmox, I ran the following commands 

"turnkey-artisan optimize" which I think does the same as 

"sudo -u www-data /usr/bin/php8.1 /var/www/invoiceninja/artisan optimize"

on my Ubuntu Install...

It did not work ... all I got was the following repeated error in an exponentially growing laravel log. it seems like it is looking for some security credentials on "http://169.254.169.254/latest/meta-data/iam/security-credentials/" as a networking guy I can tell you that it is not a valid internet address ... and also wondering where it gets that reference from !? see log below ...

***start laravel log extract***

[2023-03-07 07:26:07] production.ERROR: Error retrieving credentials from the instance profile metadata service. (cURL error 28: Connection timed out after 1000 milliseconds (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://169.254.169.254/latest/meta-data/iam/security-credentials/) {"exception":"[object] (Aws\\Exception\\CredentialsException(code: 0): Error retrieving credentials from the instance profile metadata service. (cURL error 28: Connection timed out after 1000 milliseconds (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://169.254.169.254/latest/meta-data/iam/security-credentials/) at /var/www/invoiceninja/vendor/aws/aws-sdk-php/src/Credentials/InstanceProfileProvider.php:264)
[stacktrace]
#0 /var/www/invoiceninja/vendor/aws/aws-sdk-php/src/Credentials/InstanceProfileProvider.php(139): Aws\\Credentials\\InstanceProfileProvider->handleRetryableException()
#1 [internal function]: Aws\\Credentials\\InstanceProfileProvider->Aws\\Credentials\\{closure}()
#2 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Coroutine.php(160): Generator->throw()
#3 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(204): GuzzleHttp\\Promise\\Coroutine->_handleFailure()
#4 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(153): GuzzleHttp\\Promise\\Promise::callHandler()
#5 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/TaskQueue.php(48): GuzzleHttp\\Promise\\Promise::GuzzleHttp\\Promise\\{closure}()
#6 /var/www/invoiceninja/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(159): GuzzleHttp\\Promise\\TaskQueue->run()
#7 /var/www/invoiceninja/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(184): GuzzleHttp\\Handler\\CurlMultiHandler->tick()
#8 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(248): GuzzleHttp\\Handler\\CurlMultiHandler->execute()
#9 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(224): GuzzleHttp\\Promise\\Promise->invokeWaitFn()
#10 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(269): GuzzleHttp\\Promise\\Promise->waitIfPending()
#11 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(226): GuzzleHttp\\Promise\\Promise->invokeWaitList()
#12 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\\Promise\\Promise->waitIfPending()
#13 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Coroutine.php(67): GuzzleHttp\\Promise\\Promise->wait()
#14 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(248): GuzzleHttp\\Promise\\Coroutine->GuzzleHttp\\Promise\\{closure}()
#15 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(224): GuzzleHttp\\Promise\\Promise->invokeWaitFn()
#16 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\\Promise\\Promise->waitIfPending()
#17 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Coroutine.php(103): GuzzleHttp\\Promise\\Promise->wait()
#18 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(274): GuzzleHttp\\Promise\\Coroutine->wait()
#19 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(226): GuzzleHttp\\Promise\\Promise->invokeWaitList()
#20 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\\Promise\\Promise->waitIfPending()
#21 /var/www/invoiceninja/vendor/aws/aws-sdk-php/src/AwsClientTrait.php(58): GuzzleHttp\\Promise\\Promise->wait()
#22 /var/www/invoiceninja/vendor/aws/aws-sdk-php/src/AwsClientTrait.php(86): Aws\\AwsClient->execute()
#23 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/SqsQueue.php(172): Aws\\AwsClient->__call()
#24 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(342): Illuminate\\Queue\\SqsQueue->pop()
#25 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(351): Illuminate\\Queue\\Worker->Illuminate\\Queue\\{closure}()
#26 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(160): Illuminate\\Queue\\Worker->getNextJob()
#27 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(146): Illuminate\\Queue\\Worker->daemon()
#28 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(130): Illuminate\\Queue\\Console\\WorkCommand->runWorker()
#29 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Illuminate\\Queue\\Console\\WorkCommand->handle()
#30 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#31 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure()
#32 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod()
#33 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\\Container\\BoundMethod::call()
#34 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Console/Command.php(182): Illuminate\\Container\\Container->call()
#35 /var/www/invoiceninja/vendor/symfony/console/Command/Command.php(312): Illuminate\\Console\\Command->execute()
#36 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Console/Command.php(152): Symfony\\Component\\Console\\Command\\Command->run()
#37 /var/www/invoiceninja/vendor/symfony/console/Application.php(1020): Illuminate\\Console\\Command->run()
#38 /var/www/invoiceninja/vendor/symfony/console/Application.php(312): Symfony\\Component\\Console\\Application->doRunCommand()
#39 /var/www/invoiceninja/vendor/symfony/console/Application.php(168): Symfony\\Component\\Console\\Application->doRun()
#40 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Console/Application.php(102): Symfony\\Component\\Console\\Application->run()
#41 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(155): Illuminate\\Console\\Application->run()
#42 /var/www/invoiceninja/artisan(37): Illuminate\\Foundation\\Console\\Kernel->handle()
#43 {main}

***end laravel log extract***

Anyhow the difference is between TKL install & my Ubuntu is

That we never go through the "Setup phase" (see below) where the database gets seeded and a key is obtained, which I assume is part of your installation script?

Change the owner to www-data.
sudo chown www-data:www-data /var/www/invoiceninja/.env

Generate a unique app key for your InvoiceNinja installation.
sudo php8.1 /var/www/invoiceninja/artisan key:generate

Migrate the database.
sudo php8.1 /var/www/invoiceninja/artisan migrate:fresh --seed

Here is the question, do I have to generate a key for my installation as well? ... I did that ... then and seeded the database as well ... big mistake 😫 after that I could not connect to the database with the credentials that I created  in the TKL install script..

It is a bit frustrating ...  but enjoying the challenge ...

btw ... thank you again for doing what you do at TKL ... 

Kind regards

 

George V

Jeremy Davis's picture

Firstly re your comment:

"turnkey-artisan optimize" which I think does the same as "sudo -u www-data /usr/bin/php8.1 /var/www/invoiceninja/artisan optimize" on my Ubuntu Install...

Yes, exactly! Actually it is a slightly different command, but does the same thing. FWIW 'turnkey-artisan COMMAND' runs:

runuser www-data -s /bin/bash -c "php /var/www/invoiceninja/artisan COMMAND"

Anyway, the error message you show looks to me like it's trying to read security credentials for an IAMs user or role!? FWIW the http://169.254.169.254 address is only available locally from an AWS (or Azure, probably others) server instance and provides access to AWS (Azure, etc) related meta data.

At first I was a little alarmed that it may be something malicious trying to steal credentials but a bit of a google turned up a very relevant thread (on Invoice Ninja forums - actually, there's another here too). That makes me feel confident that it's a config issue, not something malicious. The thread suggests that it may be a config issue to do with supervisord? And looking at our default supervisor config, I'm almost certain that's the issue! It's weird that I can't reproduce the issue that you're hitting (more on that below) but I'm still fairly confident that's the issue (or at least part of it).

As something of an aside, unless you have data already, I'd recommend launching a fresh instance as we've recently released TurnKey Invoice Ninja v17.3 (although it looks like we might be releasing a v17.4 Invoice Ninja appliance soon anyway...).

If you do have data already, then installing the PDF dependencies and tweaking the cron job is pretty easy. Let me know if you'd rather go that way and I can give some more details.

Also, I'm not 100% sure if it's related, but I found another bug that needs to be fixed... :(

I noticed (and fixed) that bug locally before I did anything else, so there is a possibility that that is related? I wouldn't have thought so, but I can't reproduce your error. I copied the setting you noted in your post verbatim (and notwithstanding the missing quote on the MAIL_FROM_ADDRESS line) mine worked fine?! (Even though those settings are clearly never going to work to send mail).

root@JED-TEST-inv-ninja ~# turnkey-artisan optimize

   INFO  Caching the framework bootstrap files.  

  config ................................................................................................................................. 71ms DONE
  routes ................................................................................................................................ 233ms DONE

Although if you check supervisor, you can see that I fell for the same thing re the sqs (as noted in the forum thread I linked to above). So it seems likely that that is the cause of this particular issue that you're hitting. TBH, I have no idea why it appears to "just work" for me regardless though. Very strange!

After making the adjustment (removing 'sqs' from /etc/supervisor/conf.d/turnkey-ninja.conf & restarting supervisor - see the bug report I just posted for a one-liner to do that) it all continues to work?! But at least it doesn't include the 'sqs' now (like it's not intended to apparently).

As I can't reproduce your error (even though I appear to have the same wrong config) I can only assume that that must be the issue. So I guess that TurnKey Invoice Ninja v17.4 will be coming out soon (next week or 2)...

Please let me know how you go. Especially if removing the 'sqs' from the supervisor conf.

George3430's picture

Hi Jeremy,

 

No it did not work ... and it grows the laravel log with with one entry of the log below approx ever 80 secs !!!?

[2023-03-09 05:20:02] production.ERROR: Error retrieving credentials from the instance profile metadata service. (cURL error 28: Connection timed out after 1001 milliseconds (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://169.254.169.254/latest/meta-data/iam/security-credentials/) {"exception":"[object] (Aws\\Exception\\CredentialsException(code: 0): Error retrieving credentials from the instance profile metadata service. (cURL error 28: Connection timed out after 1001 milliseconds (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://169.254.169.254/latest/meta-data/iam/security-credentials/) at /var/www/invoiceninja/vendor/aws/aws-sdk-php/src/Credentials/InstanceProfileProvider.php:264)
[stacktrace]
#0 /var/www/invoiceninja/vendor/aws/aws-sdk-php/src/Credentials/InstanceProfileProvider.php(139): Aws\\Credentials\\InstanceProfileProvider->handleRetryableException()
#1 [internal function]: Aws\\Credentials\\InstanceProfileProvider->Aws\\Credentials\\{closure}()
#2 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Coroutine.php(160): Generator->throw()
#3 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(204): GuzzleHttp\\Promise\\Coroutine->_handleFailure()
#4 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(153): GuzzleHttp\\Promise\\Promise::callHandler()
#5 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/TaskQueue.php(48): GuzzleHttp\\Promise\\Promise::GuzzleHttp\\Promise\\{closure}()
#6 /var/www/invoiceninja/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(159): GuzzleHttp\\Promise\\TaskQueue->run()
#7 /var/www/invoiceninja/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(184): GuzzleHttp\\Handler\\CurlMultiHandler->tick()
#8 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(248): GuzzleHttp\\Handler\\CurlMultiHandler->execute()
#9 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(224): GuzzleHttp\\Promise\\Promise->invokeWaitFn()
#10 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(269): GuzzleHttp\\Promise\\Promise->waitIfPending()
#11 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(226): GuzzleHttp\\Promise\\Promise->invokeWaitList()
#12 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\\Promise\\Promise->waitIfPending()
#13 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Coroutine.php(67): GuzzleHttp\\Promise\\Promise->wait()
#14 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(248): GuzzleHttp\\Promise\\Coroutine->GuzzleHttp\\Promise\\{closure}()
#15 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(224): GuzzleHttp\\Promise\\Promise->invokeWaitFn()
#16 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\\Promise\\Promise->waitIfPending()
#17 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Coroutine.php(103): GuzzleHttp\\Promise\\Promise->wait()
#18 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(274): GuzzleHttp\\Promise\\Coroutine->wait()
#19 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(226): GuzzleHttp\\Promise\\Promise->invokeWaitList()
#20 /var/www/invoiceninja/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\\Promise\\Promise->waitIfPending()
#21 /var/www/invoiceninja/vendor/aws/aws-sdk-php/src/AwsClientTrait.php(58): GuzzleHttp\\Promise\\Promise->wait()
#22 /var/www/invoiceninja/vendor/aws/aws-sdk-php/src/AwsClientTrait.php(86): Aws\\AwsClient->execute()
#23 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/SqsQueue.php(172): Aws\\AwsClient->__call()
#24 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(342): Illuminate\\Queue\\SqsQueue->pop()
#25 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(351): Illuminate\\Queue\\Worker->Illuminate\\Queue\\{closure}()
#26 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(160): Illuminate\\Queue\\Worker->getNextJob()
#27 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(146): Illuminate\\Queue\\Worker->daemon()
#28 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(130): Illuminate\\Queue\\Console\\WorkCommand->runWorker()
#29 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Illuminate\\Queue\\Console\\WorkCommand->handle()
#30 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#31 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure()
#32 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod()
#33 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\\Container\\BoundMethod::call()
#34 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Console/Command.php(182): Illuminate\\Container\\Container->call()
#35 /var/www/invoiceninja/vendor/symfony/console/Command/Command.php(312): Illuminate\\Console\\Command->execute()
#36 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Console/Command.php(152): Symfony\\Component\\Console\\Command\\Command->run()
#37 /var/www/invoiceninja/vendor/symfony/console/Application.php(1020): Illuminate\\Console\\Command->run()
#38 /var/www/invoiceninja/vendor/symfony/console/Application.php(312): Symfony\\Component\\Console\\Application->doRunCommand()
#39 /var/www/invoiceninja/vendor/symfony/console/Application.php(168): Symfony\\Component\\Console\\Application->doRun()
#40 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Console/Application.php(102): Symfony\\Component\\Console\\Application->run()
#41 /var/www/invoiceninja/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(155): Illuminate\\Console\\Application->run()
#42 /var/www/invoiceninja/artisan(37): Illuminate\\Foundation\\Console\\Kernel->handle()
#43 {main}
"}

See size of the log below 15 min after install

[image removed]

in 5 min's it grows from 979KB to 12,791KB see below

[image removed]

I edited and removed sqs see below

nano /etc/supervisor/conf.d/turnkey-ninja.conf

[program:invoice-ninja-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/invoiceninja/artisan queue:work  --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=8
redirect_stderr=true
stdout_logfile=/var/www/invoiceninja/worker.log
stopwaitsecs=3600

below is a "security adjusted" copy of my .env file

APP_NAME='TurnKey Invoice Ninja'
APP_ENV=production
APP_KEY=base64:Ne69i4Y+RbW4ERXuX9bj1nfUojIxVepHw0Ar4/SKEkk=
APP_DEBUG="false"

APP_URL=http://10.254.254.136

DB_CONNECTION="mysql"
MULTI_DB_ENABLED=false

DB_HOST="localhost"
DB_DATABASE="ninja"
DB_USERNAME="ninja"
DB_PASSWORD=43687455f6aba511f591848c4de2b1d6 ***(i dont recognise this)*** maybe encrypted with TKL tools
DB_PORT="3306"

DEMO_MODE=false

BROADCAST_DRIVER=log
LOG_CHANNEL=stack
CACHE_DRIVER=file
QUEUE_CONNECTION=database
INTERNAL_QUEUE_ENABLED=false
SESSION_DRIVER=file
SESSION_LIFETIME=120

REDIS_HOST=127.0.0.1
REDIS_PASSWORD=null
REDIS_PORT=6379

#MAIL_MAILER=sendmail
#MAIL_HOST=localhost
#MAIL_PORT=25
#MAIL_USERNAME=""
#MAIL_PASSWORD=""
#MAIL_ENCRYPTION="tls"
#MAIL_FROM_ADDRESS='ninja@10.254.254.136'
#MAIL_FROM_NAME='TurnKey Invoice Ninja'

MAIL_MAILER="smtp"
MAIL_HOST="mail.somedomain.com.au"
MAIL_PORT="465"
MAIL_USERNAME="admin@somedomain.com.au"
MAIL_PASSWORD="letmeinprettyplease"
MAIL_ENCRYPTION="ssl"
MAIL_FROM_ADDRESS="admin@somedomain.com.au"
MAIL_FROM_NAME="The Boss at Invoice Ninja"


POSTMARK_API_TOKEN=
REQUIRE_HTTPS="true"

GOOGLE_MAPS_API_KEY=
ERROR_EMAIL=
TRUSTED_PROXIES=

NINJA_ENVIRONMENT="selfhost"

#options - snappdf / phantom / hosted_ninja
PDF_GENERATOR=snappdf

PHANTOMJS_KEY='a-demo-key-with-low-quota-per-ip-address'
PHANTOMJS_SECRET=secret

UPDATE_SECRET=secret

DELETE_PDF_DAYS=60
DELETE_BACKUP_DAYS=60

COMPOSER_AUTH='{"github-oauth": {"github.com": "${{ secrets.GITHUB_TOKEN }}"}}'

GOOGLE_PLAY_PACKAGE_NAME=
APPSTORE_PASSWORD=

MICROSOFT_CLIENT_ID=
MICROSOFT_CLIENT_SECRET=
MICROSOFT_REDIRECT_URI=

APPLE_CLIENT_ID=
APPLE_CLIENT_SECRET=
APPLE_REDIRECT_URI=

I'm Lost 😖

Jeremy Davis's picture

I don't know why, but for some reason the website has decided that I can't reply directly below your previous post. Anyway, this will do. [update] I'm pretty sure it was choking on your images for some reason? I removed them and now the page seems to load properly?! FWIW we have plans to replace our current forum software. But no promises on when...[/update]

Thanks for the detailed response. Unfortunately I can't see your pictures for some reason, but I don't think that will matter.

I installed a fresh copy of TKL Invoice Ninja v17.3 from ISO to a KVM VM with 32GB root volume, 2 vCPU cores (the host has 8 slow, low power CPU cores) and 2GB of RAM. I can immediately reproduce what you are describing, but as soon as I adjust the supervisor conf to match yours and restart supervisor the laravel.log entries stop and the log remains clean. There has literally been no new entries since I restarted supervisor over 30 minutes ago! (now)

FWIW how I tested that was to open 2 ssh sessions to my server. In one, I ran:

tail -f /var/www/invoiceninja/storage/logs/laravel.log

That will follow the log file and show new entries. It's a super handy way to read logs real time while you do stuff in the other ssh session. In this case I made the relevant change to the /etc/supervisor/conf.d/turnkey-ninja.conf and restarted supervisor:

systemctl restart supervisor

One more log entry did scroll by as that was doing it's thing. But as soon as it returned (i.e. I got a prompt again) then there has been no new log entries. FWIW here is a screenshot (note the text I wrote into the log after supervisor had completed it's restart and the clock at the top of the screen - you'll probably want to right click "open image in new tab" or similar - I took that about 10 mins ago, but my ssh session looks the same right now).

I haven't tried making the changes to the .env yet, but I need to go get some dinner, so that will need to wait for tomorrow.

If possible can you please confirm that you definitely restarted supervisor and it still keeps logging that same error?

Simon Greer's picture

Chiming in late here, but I've just spent about six hours trying to get mail working using the LXC template. I also found the random AWS errors in the laravel log. Copying your steps to remove the sqs argument in the supervisor conf file, I also observed the laravel log instantly fall in line. I thought I'd tail the worker log whilst I was there and see if that gave any clues about the email issues:
root@invoiceninja .../www/invoiceninja# tail -f worker.log  
 2023-03-10 01:56:30 App\Listeners\Mail\MailSentListener ............ RUNNING
 2023-03-10 01:56:30 App\Listeners\Quote\QuoteEmailActivity . 2,952.07ms DONE
 2023-03-10 01:56:30 App\Listeners\Mail\MailSentListener ....... 25.97ms DONE
 2023-03-10 01:56:30 App\Listeners\Activity\CreatedQuoteActivity  3,100.11ms DONE
 2023-03-10 01:56:31 App\Jobs\Entity\CreateEntityPdf ........ 3,780.33ms DONE
 2023-03-10 01:56:32 App\Jobs\Entity\EmailEntity ............ 4,489.84ms DONE
 2023-03-10 01:56:32 App\Jobs\Mail\NinjaMailerJob ................... RUNNING
 2023-03-10 01:56:33 App\Jobs\Mail\NinjaMailerJob ........... 1,349.69ms DONE
 2023-03-10 01:56:33 App\Listeners\Mail\MailSentListener ............ RUNNING
 2023-03-10 01:56:33 App\Listeners\Mail\MailSentListener ....... 24.89ms DONE
 2023-03-10 02:00:03 App\Jobs\Util\ReminderJob ...................... RUNNING
 2023-03-10 02:00:03 App\Jobs\Util\ReminderJob .................. 4.59ms DONE
 2023-03-10 02:00:03 App\Jobs\Ninja\QueueSize ....................... RUNNING
 2023-03-10 02:00:03 App\Jobs\Ninja\QueueSize ................... 2.35ms DONE
Sure enough the log was speaking the truth and my inbox was full of test++1 emails. Not quite sure what that very minor configuration change did, but everything seems hunkydory now! Cheers - would never have figured that out!  
George3430's picture

Hi Jeremy,

I don't know what exactly fixed it ... I did edit turnkey-ninja.conf and restarted the supervisor yesterday and nothing changed .... the laravel log was growing exponentially like someone was feeding it "GME Genetically Modified Errors" and I could not send any emails ... 

So this morning I got up and did this .. forcefully hit the power button on my Proxmox host  (helped with the frustration I was feeling)... after shutting it down gracefully 😇 ... and restarted the proxmox box

  1. Rolled back to a snapshot straight after first install
  2. Did the security updates as per your advice/fix
  3. Edited turnkey-ninja.conf and restarted the supervisor
  4. Executed "chown www-data:www-data /var/www/invoiceninja/.env" just to be sure I don't have a rights issue
  5. Executed "sudo -u www-data /usr/bin/php8.1 /var/www/invoiceninja/artisan optimize" and  "turnkey-artisan optimize" .. as the Irish would "say to be sure to be sure" 🤣
  6. Then I fired up my browser and connected to TKL Inv Ninj ... and it came up like it did before..
  7. The 17.2 image ships with version 5.5.48 and when I fired it up the first couple of days ago, the update was to 5.5.80 ... but today the latest was 5.5.86 so I updated to the latest version and it worked

So .. this sorta leaves me hanging and a little perturbed ... what fixed it ❗❓ ... I had a look at the change logs although from 5.5.80 - 86 there seem to be a lot changes but the one that stood out for me was

"v5.5.80 What's Changed Fixes for oversized mailables"

Anyway ... I don't know, if that was an issue, why did the Ubuntu install work❓ ...

I'm happy I can now move forward with the project I'm working on ...

Greatly Relieved...

 

PS .. In regards to my project ... I was wondering if there is a way I could contact you by email but not on a public forum ... I'm quite sure you can find my email details from my registration details on this forum... If you don't mind pls drop me a line and I will explain...

Regards

George V 

Jeremy Davis's picture

Glad to hear that it sounds like you're up and running now.

Happy to chat via email. I've sent you an email. hopefully chat more soon.

Add new comment