In a fictitious world where I started my own company and developed a LEMP-stack chat application, this is how I would do it in 2016:
I would use WebSockets as the protocol in which data is being sent to and from the server. WebSockets has built-in support by nginx (since version 1.3). For caching, I would use the
expires max; function of nginx for my static content (images, css, js, fonts, etc). I would also ensure static content was named in the scheme of
name-md5hash.ext. This would allow the caching on the client side to shine.
I would ensure my php files are gzip compressed (with .gz extension) to allow fast transferring of data between client and server. Nginx finds the .gz files when gzip compression is turned on and uses them if they exist. I would design small php-fpm servers that can support ~2,000 users at a time (basically, a dual core processor with 4gb of ram). This would allow ease of scaling at lower costs.
I would use GlusterFS for fault-tolerant storage. I would use the
mount -t gluster option on the servers as it would allow distributed nfs capability for storage of uploaded content.
I would use MariaDB MaxScale as the database as it would allow for ease of scaling.
In short, it would look similar to the following diagram: