Operation guidelines

This applies to RedHat family Linux distributions (RedHat, CentOS, Fedora, ...) and to ythe Tomcat® application server. It can easily be transposed to other technical platforms.

Services restart

When required the involved services may need to be stopped in the following order:

Then clean up the Tomcat work folders (and optionally the technical logs folder):

rm -fr $TOMCAT_ROOT/work/Catalina $TOMCAT_ROOT/conf/Catalina
rm -fr $TOMCAT_ROOT/logs/*

Then the services are to be restarted in the following order:

Logs review

Technical logs

The Tomcat server technical logs are located in $TOMCAT_ROOT/logs.

The content of these logs is managed at Tomcat configuration level, please refer to Tomcat documentation for details (e.g. this document for Tomcat 8)

Application logs

The application-specific technical logs are located in $TOMCAT_ROOT/data/simplicite/<my application>/log/ (classical packaging) or in $TOMCAT_ROOT/webapps/ROOT/WEB-INF/log (sandbox packaging) or in custom location depending on your installation

The content of these logs is managed by Log4J and can be customized by overriding the default log4j.xml file provided in the confSimplicite-x.y.jar (classical packaging) or in $TOMCAT_ROOT/webapps/ROOT/WEB-INF/classes (sandbox packaging)

Database logs

Some application log events are recording logs entries into the m_log table of the default database. Make sure to purge this table on a regular basis (Operation > Logs > and use the Delete list list action)

Save and restore

For a given application a comprehensive save consist in:

These two operations needs to be done exactly at the same time to avoid any data inconsistencies.

To restore the application, the database dump and the archive must be restored in their initial locations.

Note: as of version 3.2 the documents can be stored in the database as BLOBs, in this case saving the database is sufficient (no need to save the document data directory)

Monitoring

All technical components may need to be monitored (especially Tomcat and the database engine) using any convenient tool.

Application-level tools are available (from the UI with an operator profile or from command line) to do basic technical and applicative monitoring. Typically the health check page/service can be called and parsed on a regular basis:

curl -b cookies.txt -c cookies.txt "<base URL>/health[?format=json]"

Note: the -b and -c argument are required to reuse the same Tomcat session

Typical output is:

{
  "platform": {
    "status": "OK",
    "version": "3.2.M05",
    "builton": "2016-10-12",
    "encoding": "UTF-8",
    "systemdate": "2016-10-13 11:51:04"
  },
  "application": {
    "applicationversion": "1.0.0",
    "contextpath": "",
    "contexturl": "<base URL>",
    "activesessions": 1,
    "enabledusers": 12,
    "totalusers": 14
  },
  "os": {
    "name": "Linux",
    "architecture": "amd64",
    "version": "3.10.23-xxxx-grs-ipv6-64",
    "systemencoding": "UTF-8"
  },
  "disk": {
    "diskfree": 10741,
    "diskusable": 9741,
    "disktotal": 19841
  },
  "javavm": {
    "version": "1.8.0_91",
    "vendor": "Oracle Corporation",
    "vmname": "OpenJDK 64-Bit Server VM",
    "vmversion": "25.91-b14",
    "vmscriptengine": "rhino",
    "heapfree": 117231,
    "heapsize": 210432,
    "heapmaxsize": 466432,
    "totalfreesize": 373231
  },
  "cache": {
    "grantcache": 14,
    "grantcachemax": 0,
    "grantcacheratio": 0,
    "objectcache": 10,
    "objectcachemax": 10000,
    "objectcacheratio": 0,
    "processcache": 0,
    "processcachemax": 10000,
    "processcacheratio": 0
  },
  "database": {
    "vendor": "3",
    "productname": "HSQL Database Engine",
    "productversion": "2.3.2",
    "drivername": "HSQL Database Engine Driver",
    "driverversion": "2.3.2",
    "dbdate": "2016-10-13 11:51:04",
    "dbdateoffset": 0
  },
  "healthcheck": {
    "date": "2016-10-13 11:51:04",
    "elapsedtime": 2
  }
}

If this page does not return response or returns a HTTP status different from 200 or if the response value for platform.status is not OK, something needs to be investigated/fixed.

Others

You should also review and purge on a regular basis:

System updates

System updates must be applied regularly, especially in case of critical security updates.

On Linux CentOS, you can check if there are pending system upgrades packages by:

sudo yum check-update

You can then apply the updates by:

sudo yum update

You should then clean up update packages by:

sudo yum clean all

If there is an OpenJDK update you must stop the running Tomcat instances prior to the update and restart them after the update.

If there are other running services impacted by the updates (e.g. database service), you must restart the services.

If there is a kernel update you must reboot the server after the update.

To clean up old kernels you can do:

sudo yum install yum-utils
sudo package-cleanup --oldkernels --count=2

Tomcat updates

The Tomcat server must be updated regularly, depending on the way it has been installed the processus may vary.

Platform updates

The Simplicit√©® platform must be updated regularly, at least on its maintenance branch (see versions), depending on the way it has been installed the processus may vary.