Tony Upson's picture

Overview:

Version: OTRS v3.3.18
Mail Service: Microsoft Exchange 2013 CU15 2 Node DAG
DNS: A record that talks to the DAG IP Address (which is supporting two different Exchange nodes)
OTRS Certificate: Issued an internal Microsoft CA certificate replacing self-signed certificate
Desktop: Windows 10 (1803) 64-Bit (IPv6 NIC setting disabled)

Problem:

When responding to a ticket, after hitting submit - some times the window sits there for up to 60 seconds before it completing the submission and sometimes its instant. Both in Internet Explorer and Google Chrome.

I am trying to figure out why this delay is occurring and if there is anywhere logging this?

Forum: 
Tony Upson's picture

I decided to manually install CENTOS7 and OTRS6 from scratch. Thanks anyway.

- Tony

Network Engineer

Federal Gov't

Jeremy Davis's picture

Apologies on delay responding. TBH, I'm not super familiar with OTRS so I'm not sure how much direct help I can be, but I do know TurnKey pretty well.

Even though you've moved on, I'll reply as best I can, just in case you come back to it, and/or someone else hits this issue, then this will give something of a troubleshooting starting point.

The TurnKey OTRS appliance installs OTRS from the Debian repositories. So if you are having issues, it's often worth checking to see if there is an updated package. FWIW TurnKey auto installs security updates, but sometimes bugfix packages are released - which aren't auto-installed. This appears to be relevant for the v15.0 appliance (YMMV on older versions).

More generally, OTRS is written in Perl, and is run as a CGI script via Apache. So the Apache logs may provide some insight into issues within OTRS itself. You'll find Apache logs in /var/log/apache2.

Also, my googling turned up a previous post by you suggesting that more detailed OTRS logs can be found within /var/lib/otrs/. As this was quite an old post, I'm not 100% sure that it still applies, although I'll check it out and if it does, then I might add a convenience symlink from /var/log/otrs to the next OTRS appliance release, to make it easier to find those OTRS logs.

I forget whether OTRS itself handles the outgoing mail (SMTP) directly, or whether it leverages Postfix (probably the latter - at least by default). If it uses Postfix it may also be worth checking the mail logs too (/var/log/mail.log & /var/log/mail.info).

Another thing to consider is that an intermittent delay like that may be caused by something else. Perhaps a DNS lookup is being slow? If your SMTP server has a static IP, it may be worth adding an entry to the local hosts file of your server to test if that improves things. If it does, then a DNS lookup delay seems highly likely.

It may also be worth monitoring the RAM and CPU usage of your server when the delay occurs vs when it doesn't. It may be that your server needs a bit more RAM, or perhaps needs some tuning to reduce the background load, or it could be that there is a memory leak within the application itself. FWIW the version of OTRS preinstalled in the v15.0 appliance does appear to have a memory leak and the newer package in the Debian repos appears to resolve that. Although I'm not at all sure if that is relevant to your experience.

Regardless, good luck with it all.

Jeremy Davis's picture

Whilst answering another email related question, I just came across some interesting information that seemed possibly relevant and thought that I'd add it here. Apparently sometimes SMTP servers can delay sending of mail (often as a anti-spam measure, but also if it's busy), and how that may affect an application trying to send mail will depend on the design and/or configuration of the application itself (i.e. OTRS in this case).

By default, most applications will wait for the SMTP server to respond that it has successfully sent the mail (or not as the case may be). However, to work around possible delays this may cause, some applications support "asynchronous sends" i.e. some sort of outgoing mail queue - where the application does not wait for the SMTP server to confirm that the mail has been sent. I'm not 100% sure if this also applies to OTRS, but it seems likely.

I assume that by default the TurnKey appliance is configured to send via local Postfix, which I assume would mitigate the issue. I.e. Postfix would queue the mail if it couldn't send immediately, allowing OTRS to continue. However, if you are using a remote SMTP server directly from OTRS, then perhaps your SMTP server is intermittantly slow to send the emails, thus making OTRS pause until the mail sends?

If you are using SMTP direct from OTRS, it might be worth re-configuring it to use Postfix, then configure Postfix to leverage your SMTP server as a mail relay. Hopefully that would resolve the issue (assuming that the issue is SMTP server causing the delays). Otherwise, some troubleshooting on the SMTP server may be worth consideration? (Although in fairness, up to 60 seconds sounds excessive!)

Tony Upson's picture

Thanks for the follow up. Yes, I have configured OTRS to directly talk to my SMTP Relay Servers, so as you mentioned may be the delay on waiting for the system to accept more connections.

Funny you mentioned Postfix, I actually have that configured; assuming it was using it anyway; even when I modified the SMTP settings in OTRS, but it appears I am bypassing those configurations.

If it gets too bothersome, perhaps I can test that out. How do I change it back to use Postfix? In OTRS I can either use Sendmail or SMTP? I assume changing the host from my SMTP server to the help desk appliance host name within SendmailModule::Host would do the trick?

With respect to memory, I changed the out-of-box settings immediately from 524MBs to 8GBs of RAM and 1vCPU to 2vCPUs. For it's use, I feel like my modifications are overkill and prepared to handle its help desk inquiries.

 

 

- Tony

Network Engineer

Federal Gov't

Jeremy Davis's picture

By my understanding when you directly configure an app (such as OTRS) to use a remote SMTP server it will bypass the local Postfix config and use the remote SMTP server directly to send mail.

Although having Postfix configured is still worthwhile IMO (even if that too is configured to use a remote SMTP to actually send) as then other services which send emails (such as backup and security update notifications) should still get sent.

Regarding OTRS specifically, from my understanding, setting "sendmail" as the outgoing SMTP mail server should do the trick (FWIW Postfix provides a sendmail binary).

I'm sure that 8GBs of RAM and 2vCPUs should be more than enough! :)

Tony Upson's picture

I see the v15.0 appliance has OTRS5x installed. Before knowing how to deploy OTRS on a linux platform, I would have loved to take advantage of going from v3 to v5, but I found a blog that provided step-by-step instructions of preparing CENTOS7 with OTRS6, so I took advantage of the opportunity to be current with OTRS support.

I am all for the security parameters Turnkey Linux takes into account with your appliances, but I got sick of the OTRS forums barking up my tree about using unsupported version(s) [as you experienced with me previously]. Even with this recent request you replied to, that was the first thing they mentioned (you are using an unsupported version, upgrade).

So I bit the bullet and took about 8hrs to document every step of the way and the process went smooth. I had to find the differences of locations of the /otrs/ files and directories as it is far different than where you guys install it, so that was pretty much my delay in configuration.

I also will have to lose my ticket history with this migration, as I do not want to have to upgrade from v3 to v4 to v5 to v6 just to keep a history of 1000 tickets. This will just be a fresh transition to current technology in my opinion. I'll simply export the ticket history into a PDF and save it (as I doubt we'll ever need to use it though); in our environment. 

Thanks for taking the time to chat on it though, I always appreciate your willingness to respond.

- Tony

Network Engineer

Federal Gov't

Jeremy Davis's picture

It's a really unfortunate situation between OTRS upstream and Debian with regard to the Debian OTRS packages. In my experience with other software, when Debian and upstream can work together for the benefit of users, everybody wins. But it seems that the relationship between OTRS and Debian is somewhat antagonistic.

In fairness to OTRS, I can understand that the fact that the OTRS packages feel so far behind their supported versions for a while would not have been great for them. (AFAIK at some point the Debian OTRS package maintainer went AWOL causing the significant version drift). I imagine that would have been especially frustrating if issues that they had fixed (in newer versions) kept coming up in their support forums. Also, as a software developer, I can understand that ideally you'd like your users to be getting the benefits of your latest releases (whether they be bugfixes, added features, usability and/or UI improvements, etc).

Although from the other side (and essentially TurnKey's perspective), the option of automated (backported) security updates with (relative) assurance of non-breaking changes, with very limited application changes is also very appealing. As I say, if they could work together, would be ideal...

FWIW, I did consider moving the TurnKey appliance to an upstream install (rather than using the Debian packages) but when I saw that Debian 9/Stretch (basis of v15.x) had OTRS v5 I relented and stuck with the Debian packages (for all the afore-mentioned advantages). FWIW v6 is actually in stretch-backports and will be in the upcoming 10/buster release (likely the basis of TurnKey v16.x).

Anyway, thanks for your kind words. Sorry to see you go, but I understand the frustrations. Good luck with things moving forward.

Add new comment