NameNode Analytics 1.6.7

• [#287] fileTypes.html not producing correct results by @pjeli in #288
• [#290] Add admin API to kill running queries by @pjeli in #291
• [#230] Investigate using different Collections libraries for filter speed-up by @pjeli in #292
• [#293] OOME when using INodeCollectionFilter by @pjeli in #295
• Add support for HDFS 2.10.0 by @shvachko in #300
• [#306] Add simple ContentSummary call by @pjeli in #307
• [#308] Introduce AirConcurrentMap as possible backing HashMap by @pjeli in #309
• [#310] CachedQuotas.analyze creates unnecessary HashSets by @pjeli in #313

Full Changelog: 1.6.6...1.6.7

NameNode Analytics 1.6.6

The thirteenth release of NameNode Analytics.

This is both a performance and feature release that adds two-level histograms, cached quota computation parallelization and includes the following:
[#285] Add query links for finding directories with no quotas
[#283] Add file type histogram to SuggestionsEngine
[#281] Improve cached quotas by using two level histograms
[#279] Provide histogram.html page
[#276] Add display for how many transactions NNA is behind
[#275] Provide easier restart functionality
[#267] Multi-level grouping in Stream API

It is recommended to update to this release as it has some serious performance improvements.

I recommend checking out the new /info information, the new auto-fetch image on restart functionality, the /fileTypes.html page, and playing around with the new /histogram.html and /histogram2.html pages.

Supported OS:
-CentOS/RedHat
-Ubuntu

Supported Java:
-JDK8+

Documentation here: https://nnanalytics.readthedocs.io/en/release-1.6.6/

NameNode Analytics 1.6.5

The twelfth release of NameNode Analytics.

This is a feature release to 1.6.4 that focuses on adding an SQL-like API and includes the following:
[#270] Allow setting of user variables in SQL page
[#269] Provide error message output on SQL page
[#266] Parse accessTime and modTime dates for SQL queries
[#262] Dump SQL response into Datatables
[#263] Support ASC and DESC sorting in SQL command
[#260] SQL aggregation functions not working
[#258] Add a SQL-like API
[#249] Add pointer to trenner/ahar in Suggestions page
[#250] Dump endpoint can cause Error
[#247] Add dirNumChildren summation for dirs set sums
[#245] Add more file types

It is recommended to update to this release as it fixes some bugs.

Supported OS:
-CentOS/RedHat
-Ubuntu

Supported Java:
-JDK8+

Documentation here: https://nnanalytics.readthedocs.io/en/release-1.6.5/

NameNode Analytics 1.6.4

The eleventh release of NameNode Analytics.

This is a quick maintenance release to 1.6.3 that addresses the following:
[#243] File type histogram not detecting app logs
[#237] FileTypeHistogram not detecting part files
[#209] Add an [Explain] button for Totals

If you do not need any of the fixes above then do not be concerned.
Feel free to remain on 1.6.3.

Supported OS:
-CentOS/RedHat
-Ubuntu

Supported Java:
-JDK8+

Documentation here: https://nnanalytics.readthedocs.io/en/release-1.6.4/

NameNode Analytics 1.6.3

This is the tenth release of NameNode Analytics! 🎈 🎂

[#235] Update README.md with links and images
[#223] Externalize web resources
[#232] Add more file types into FileTypeHistogram
[#229] Actual numbers report for Space and Namespace quotas
[#224] Speed-up JavaStreamQueryEngine with better stream utilization
[#226] Add progress info for background analysis

It is HIGHLY recommended to update to this version.

Major changes were done to the QueryEngine to deal with performance at large scale (400M+ inodes). As a result, you should see anywhere between 4-10x speedup on all real time queries and SuggestionEngine report times. JavaCollectionQEngine has been removed as a result.

If you are upgrading from any previous line, please be sure to rename your security.properties file to application.properties as the file NNA looks for when bootstrapping was deprecated as part of #206.

If you are upgrading from a 1.5.x or 1.4.x line, please be sure to modify your /usr/local/nn-analytics/bin/nna_env file to override the NNA_MAIN export to point to org.apache.hadoop.hdfs.server.namenode.analytics.WebServerMain as opposed to com.paypal.namenode.WebServerMain as the com.paypal packages were removed as part of #175.

Supported OS:
-CentOS/RedHat
-Ubuntu

Supported Java:
-JDK8+

Documentation here: https://nnanalytics.readthedocs.io/en/release-1.6.3/

NameNode Analytics 1.6.2

The ninth release of NameNode Analytics.

Most of the effort this time was focused around refactoring the QueryEngine and making it easier to read / follow. Please be sure to check out the cached reports feature available via the UI.

[#220] Split UI into Queries and Cached Reports and expose cached reports
[#218] Debian builds attempt to make use of chkconfig
[#217] Refactor Query Engine code
[#216] Improve code coverage
[#211] QueryEngine code duplication reduction
[#206] Rename SecurityConfiguration to ApplicationConfiguration
[#207] Modified INFO log messages to DEBUG

It is recommended to update to this version.

If you are upgrading from any previous line, please be sure to rename your security.properties file to application.properties as the file NNA looks for when bootstrapping was deprecated as part of #206.

If you are upgrading from a 1.5.x or 1.4.x line, please be sure to modify your /usr/local/nn-analytics/bin/nna_env file to override the NNA_MAIN export to point to org.apache.hadoop.hdfs.server.namenode.analytics.WebServerMain as opposed to com.paypal.namenode.WebServerMain as the com.paypal packages were removed as part of #175.

Supported OS:
-CentOS/RedHat
-Ubuntu

Supported Java:
-JDK8+

Documentation here: https://nnanalytics.readthedocs.io/en/release-1.6.2/

NameNode Analytics 1.6.1

The eighth release of NameNode Analytics.

[#203] Add finer grained logging of time spent in Suggestions queries
[#195] Ignore the namespace quota on root directory
[#194] Reduce output size when using suggestions?all API
[#190] Add estimation for when next NNA report will arrive
[#193] Increase sleep time for restart function

It is recommended to update to this version.

If you are upgrading from a 1.5.x or 1.4.x line, please be sure to modify your /usr/local/nn-analytics/bin/nna_env file to override the NNA_MAIN export to point to org.apache.hadoop.hdfs.server.namenode.analytics.WebServerMain as opposed to com.paypal.namenode.WebServerMain as the com.paypal packages were removed as part of #175.

Supported OS:
-CentOS/RedHat
-Ubuntu

Supported Java:
-JDK8+

Documentation here: https://nnanalytics.readthedocs.io/en/release-1.6.1/

NameNode Analytics 1.6.0

The seventh release of NameNode Analytics.

[#188] Improvements to cached queries
[#184] Allow users to cache queries for report generation
[#185] Modify filesystem object size in memoryConsumed query
[#183] Make main method public for WebServerMain
[#181] Provide a filter for check if an inode is under a quota
[#179] Provide a way to get the raw timestamps from CSV output Histogram
[#177] Introduce new HTTP Server based on Hadoop web server
[#175] Remove com.paypal packages
[#173] Quota tables HTML page has wrong order

It is recommended to update to this version.

If you are upgrading from a 1.5.x or 1.4.x line, please be sure to modify your /usr/local/nn-analytics/bin/nna_env file to override the NNA_MAIN export to point to org.apache.hadoop.hdfs.server.namenode.analytics.WebServerMain as opposed to com.paypal.namenode.WebServerMain as the com.paypal packages were removed as part of #175.

We have dropped support for Hadoop versions 2.4.x, 2.5.x, and 2.6.x. If you need updates from the NNA 1.6.x line back-ported to support one of these Hadoop versions please raise an issue or message @pjeli on the NNA Gitter. We will be dropping support for 2.x entirely as NNA moves closer to Apache adoption.

Please checkout the new /setCachedQuery, /getCachedQuery, and /removeCachedQuery APIs via the ReadTheDocs link below. This API is highly encouraged to use over querying API repeatedly.

Supported OS:
-CentOS/RedHat
-Ubuntu

Supported Java:
-JDK8+

Documentation here: https://nnanalytics.readthedocs.io/en/release-1.6.0/

NameNode Analytics 1.5.3

The sixth release of NameNode Analytics.

This release focuses on simple bug-fixes in the 1.5.X line:
[#169] Suggestions page rendering bad token dates - 9e7c2c9
[#171] Support passwordless authentication - 0cd6dc7

It is recommended to update to this version.

Supported OS:
-CentOS/RedHat
-Ubuntu

Supported Java:
-JDK8+

Documentation here: https://nnanalytics.readthedocs.io/en/release-1.5.3/

NameNode Analytics 1.5.2

The fifth release of NameNode Analytics.

This release focuses on further UI changes and improvements in the 1.5.X line:
[#165] Allow blockSize as a filter field and find field - 6ed729f
[#163] Add quotas.html to navbar - 4c0e4ea
[#161] Increase file type discovery - e92af52
[#159] Restore ability to create Excel reports on datatables - e05f031

It is recommended to update to this version.

Supported OS:
-CentOS/RedHat
-Ubuntu

Supported Java:
-JDK8+

Documentation here: https://nnanalytics.readthedocs.io/en/release-1.5.2/