Very often we will run into large message files above 1 GB. These log files can be viewed by regular text editors. However, most text editors have a limitation of supporting files over a certain size.
In this article, we will cover how to read large message files (e.g., /var/log/messages) generated on Linux systems.
To debug issues in Cloud environments, it's essential for you to know where the log files are and what is contained in each log file. On Linux servers, over a dozen log files are located in /var/log directory. Here we only focus on one of them:
- This log aims at storing "general system activity" messages.
- There are several things that are logged in /var/log/messages including mail, cron, daemon, kern, auth, etc.
- The severity of messages could be
- Older message files are archived periodically with their name annotated with the date.
in which you can specify rules (i.e., selector + action) of logging. For example, you can log anything of level informational or higher except mail, cron, or private authentication message:
and messages are logged into a file named /var/log/messages.
Limitations of Text Editors
- 64 kilobytes (KB)
- It's said of no size limit. But, the real problem is performance.
- Depends on the version of Wordpad, some people say it can support files of size up to 20 MB without performance issues.
- It can handle file sizes up to the largest contiguous chunk of 32-bit virtual memory.
Basically, there are two solutions of dealing with large text files:
- Find a more capable text editor
- Divide and conquer
On Linux systems, a good approach is 'divide-and-conquer" by using split command like:
split -b1000m messages-20160315T2201 split-messages
After splitting, a good text editor such as Textpad will be able to read a file of 1000 MB easily.
- 20 Linux Log Files that are Located under /var/log Directory
- How to split large file into several smaller files – Linux
- "File Is Too Large to Open" Message in Notepad
- Wordpad - File size Limit ?
- Text editor to open big (giant, huge, large) text files
- Configuring and auditing Linux systems with Audit daemon
- Difference between /var/log/messages, /var/log/syslog, and /var/log/kern.log?
- Textpad specification