When upgrading to a new version of Elasticsearch, you need to upgrade each of the products in your Elastic Stack. Beats and Logstash 5.6 are compatible with Elasticsearch {version} to give you flexibility in scheduling the upgrade.
The steps you need to take to upgrade differ depending on which products you are using. Want a list that’s tailored to your stack? Try out our {upgrade_guide}[Interactive Upgrade Guide].
If you are running a pre-6.0 version, we recommend upgrading to the most recent 5.6 before upgrading to {version}. {xpack} 5.6 provides a free Upgrade Assistant that identifies issues you need to address before upgrading and simplifies migrating indices that need to be reindexed before you upgrade. The Upgrade Assistant is enabled with both Trial and Basic licenses. You can install {xpack} solely for the purpose of upgrading.
Rolling upgrades are supported when upgrading from Elasticsearch 5.6 and Elasticsearch 6.0-6.2 to {version}. Upgrading from any version prior to 5.6 requires a full cluster restart.
|
Important
|
2.x indices are not compatible with {version}. You must remove or reindex them on your 5.n cluster before upgrading to {version}. The internal Kibana and {xpack} indices and the default Beats and Logstash mapping templates also need to be updated to work with {version}. |
Before upgrading the Elastic Stack to {version}:
-
Back up your data. You cannot roll back to an earlier version unless you have a backup of your data. For information about creating snapshots, see {ref}/modules-snapshots.html[Snapshot and Restore].
-
Check the Elasticsearch {ref}/settings.html[deprecation log] to see if you’re using any deprecated features and update your code accordingly. By default, deprecation log messages are enabled at the
WARNlevel. -
Review the breaking changes for each product you use and make the necessary changes so your code is compatible with {version}:
-
{beats-ref}/breaking-changes.html[Beats {version} breaking changes]
-
{apm-server-ref}/breaking-changes.html[APM Server {version} breaking changes]
-
{ref}/breaking-changes.html[Elasticsearch {version} breaking changes]
-
{hadoop-ref}/breaking-changes.html[Elasticsearch Hadoop {version} breaking changes]
-
{kibana-ref}/breaking-changes.html[Kibana {version} breaking changes]
-
{logstash-ref}/breaking-changes.html[Logstash {version} breaking changes]
Important-
If you’re upgrading from 2.n, make sure you check the breaking changes from 2.n to 5.n, as well as from 5.n to 6.n!
-
If you are using {ml} {dfeeds} that contain discontinued search or query domain specific language (DSL), the upgrade will fail. In 5.6.5 and later, the Upgrade Assistant provides information about which {dfeeds} need to be updated.
-
-
{ref}/docs-reindex.html[Reindex] or delete any indices created on 2.n. We recommend upgrading to the most recent 5.6 and using the {xpack} Reindex Helper to reindex 2.n indices.
-
If Kibana and {xpack} are part of your stack, upgrade the internal Kibana and {xpack} indices. We recommend using the {xpack} 5.6 Reindex Helper to upgrade the internal indices. If you’re performing a full cluster restart upgrade from an earlier version, you can also use the
_xpack/migration/upgradeAPI directly to upgrade the internal indices after you install Elasticsearch {version}. -
If you use {xpack} to secure your cluster:
-
Make sure TLS is enabled to encrypt communications between nodes. TLS must be enabled to upgrade to {version}. For more information, see {stack-ov}/encrypting-communications.html[Encrypting communications].
NoteEnabling TLS requires a full cluster restart. Nodes that have TLS enabled cannot communicate with nodes that do not have TLS enabled. You must restart all nodes to maintain communication across the cluster. -
Make sure real passwords are configured for the built-in
elasticsearch,kibana, andlogstash_systemusers. They cannot use the 5.n default password (changeme). For more information, see {stack-ov}/built-in-users.html[Built-in users].
-
-
Stop any {ml} jobs that are running before starting the upgrade process. See {stack-ov}/stopping-ml.html[Stopping {ml}].
|
Important
|
Test upgrades in a dev environment before upgrading your production cluster. |
Upgrade the Elastic Stack products you use in the following order:
-
Elasticsearch Hadoop: {hadoop-ref}/install.html[install instructions]
-
Elasticsearch: {ref}/setup-upgrade.html[upgrade instructions]
-
Kibana: {kibana-ref}/upgrade.html[upgrade instructions]
-
Logstash: {logstash-ref}/upgrading-logstash.html[upgrade instructions]
-
Beats: {beats-ref}/upgrading.html[upgrade instructions]
-
APM Server: {apm-server-ref}/upgrading.html[upgrade instructions]
|
Note
|
Logstash 5.6 and 6.n and Beats 5.6 and 6.n are compatible with all 6.n versions of Elasticsearch. This provides flexibility in when you schedule the upgrades for your Logstash instances and Beats agents. We recommend upgrading Logstash and Beats as soon as possible to take advantage of performance improvements and other enhancements. |
Starting in 6.3, the default distributions of {es}, {ls}, and {kib} include {xpack} and a free Basic license that never expires.
You can perform rolling upgrades to 6.3 from OSS-only clusters running 5.6 or 6.0-6.2. {xpack} Basic features are operational once the cluster is fully upgraded. If you are already using {xpack}, your settings are preserved when you upgrade.
|
Important
|
If you use {xpack} you must explicitly remove the old {xpack} plugin
before restarting: bin/elasticsearch-plugin remove x-pack. The node will fail
to start if the {xpack} plugin is present.
|
You must explicitly enable data collection after the upgrade to use monitoring.
Set xpack.monitoring.collection.enabled to true with the _cluster/settings
API:
PUT /_cluster/settings
{
"persistent" : {
"xpack.monitoring.collection.enabled" : "true"
}
}To take all of the {xpack} features for a spin, you can start a 30-day trial from Kibana, or with the Start Trial API:
POST _xpack/license/start_trialThe 30-day trial enables you to try out the full set of Platinum features, including security, machine learning, alerting, graph capabilities, and more.
{xpack} 5.6 provides migration and upgrade APIs for Elasticsearch and a Upgrade Assistant UI for Kibana. These tools are included with the {xpack} trial license and the free Basic license.
To upgrade to {version} from 5.6:
-
{ref}/setup-upgrade.html[Upgrade Elasticsearch] to the most recent 5.6 and install {xpack} on all nodes in your cluster. If you are upgrading from an earlier 5.x release, you can perform a rolling upgrade. To upgrade from older versions you must perform a full cluster restart.
If your trial license to use {xpack} expires, register for a free Basic license. To apply the license, upload the license file with the
licenseAPI:license -d @license.json -
If {xpack} IS NOT normally a part of your {stack}, disable {security} in
elasticsearch.yml:xpack.security.enabled: false
-
Upgrade {kib} to the most recent 5.6 and install {xpack}.
-
If you disabled {security} in
elasticsearch.yml, also disable {security} inkibana.yml:xpack.security.enabled: false
-
Use the Upgrade Assistant in Kibana to view incompatibilities that you need to fix, identify any 2.x indices that need to be migrated or deleted, and upgrade the internal indices to the {major-version} index format.
You can also call the Elasticsearch migration APIs directly:
/_xpack/migration/assistance-
Runs a series of checks on your cluster, nodes, and indices and returns a list of issues that need to be fixed before you can upgrade to {version}.
/_xpack/migration/upgrade-
Upgrades the {watcher} and {security} indices to a single-type format compatible with Elasticsearch 6.x.
-
Once you’ve resolved all of the migration issues, perform a {ref}/rolling-upgrades.html[rolling upgrade] from Elasticsearch 5.6 to {version}.
It is possible to upgrade directly to {major-version} from a pre-5.6 installation, but it requires a {ref}/restart-upgrade.html[full cluster restart] and you must manually reindex any 2.x indices you need to carry forward to {major-version}.
|
Important
|
If you use Kibana or {xpack}, you also need to upgrade the internal Kibana and {xpack} indices. For information about upgrading them after you install Elasticsearch {version}, see Upgrading internal indices. |
To manually reindex a 2.x index:
-
Create an index with 6.x compatible mappings.
-
Use the {ref}/docs-reindex.html[reindex API] to copy documents from the 2.x index into the new index. You can use a script to perform any necessary modifications to the document data and metadata during reindexing.
-
Use the {ref}/indices-aliases.html[_aliases] API to add the name of the 2.x index as alias for the new index and delete the 2.x index.
The format used for the internal indices used by Kibana and {xpack} has changed in {major-version}. Before you can run Kibana and {xpack} in {version}, these indices must be upgraded to the new format. If you are upgrading from a version prior to 5.6, you must upgrade them after after installing Elasticsearch {version}.
To get a list of the indices that need to be upgraded, install {xpack} and use
the {ref}/migration-api-assistance.html[_xpack/migration/assistance API]:
GET /_xpack/migration/assistanceTo upgrade the .security index:
-
On a single node, add a temporary superuser account to the
filerealm. -
Use the {ref}/migration-api-upgrade.html[
_xpack/migration/upgrade] API to upgrade the security index, submitting the request with the credentials for the temporary superuser:POST /_xpack/migration/upgrade/.security -
Delete the temporary superuser account from the file realm.
You can use your regular administration credentials to upgrade the other
internal indices using the _xpack/migration/upgrade API.
|
Tip
|
Once you upgrade the .kibana index, you can run Kibana and use the
{xpack} Reindex Helper UI to upgrade the other indices.
|
A single click in the Elastic Cloud console can upgrade a cluster to a newer version, add more processing capacity, change plugins, and enable or disable high availability, all at the same time. During the upgrade process, Elasticsearch, Kibana, {xpack} and the officially included plugins are upgraded simultaneously.
Although upgrading your Elastic Cloud clusters is easy, you still need to address breaking changes that affect your application. Minor version upgrades, upgrades from 5.6 to {major-version}, and all other cluster configuration changes can be performed with no downtime.
To avoid downtime when a full cluster restart is required:
-
Provision an additional cluster with the new Elasticsearch version, reindex your data, and send index requests to both clusters temporarily.
-
Verify that the new cluster performs as expected, fix any problems, and then permanently swap in the new cluster.
-
Delete the old cluster to stop incurring additional costs. You are billed only for the time that the new cluster runs in parallel with your old cluster. Usage is billed on an hourly basis.
To learn more about the upgrade process on Elastic Cloud, see {cloudref}/upgrading.html[ Upgrade Versions] and {cloudref}/cluster-config.html[Configuring Elastic Cloud].
|
Note
|
Elastic Cloud only supports upgrades to released versions. Preview releases and master snapshots are not supported. |