What's New in v23.2

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image, both Intel and ARM images are generally available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach:v23.2.1

Source tag

To view or download the source code for CockroachDB v23.2.1 on Github, visit v23.2.1 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0...v23.2.1

Security updates

• Introduced the server.redact_sensitive_settings.enabled cluster setting, which is false by default. If set to true, then the values of the following settings will be redacted when accessed through SHOW commands or other introspection interfaces. In the future, any other sensitive cluster settings that are added will be redacted as well. Users who have the MODIFYCLUSTERSETTING privilege can always view the unredacted settings. #117729
  • server.oidc_authentication.client_id
  • server.oidc_authentication.client_secret
• If the server.redact_sensitive_settings.enabled cluster setting is set to true, then the MANAGEVIRTUALCLUSTER privilege is required to view the values of the per-virtual-cluster overrides for sensitive cluster settings. #117729
• The DB Console session cookie is now marked HttpOnly to prevent it from being read by any JavaScript code. #119249
• DB Console cookies are marked Secure for the browser when the cluster is running in secure mode. #119249

General changes

• Updated Go version to 1.21.3. #115339

Enterprise edition changes

• Added a new SQL function fips_ready, which can be used to verify the FIPS readiness of the gateway node. #115202
• Physical Cluster Replication (PCR) now retries for approximately 3 minutes before failing. This is increased from 20 µs. #116402
• Fixed a bug where changefeeds that targeted schema-locked tables could fail due to a very old highwater timestamp being incorrectly persisted. #117961
• Fixed a bug where creating a changefeed that targeted tables with a DECIMAL(n) column (i.e., zero-scale DECIMAL column), format='avro', and diff would cause a panic. #118895

SQL language changes

• Added the sql.ttl.default_select_rate_limit cluster setting and the ttl_select_rate_limit table storage parameter to set the TTL select rate limit. This sets the number of records per table per second per node that can be selected by the TTL job. #115801
• Fixed a bug in PL/pgSQL where altering the name of a sequence or user-defined type (UDT) that was used in a PL/pgSQL function or procedure could break them. This bug was only present in v23.2 alpha and beta releases. #116419
• Added support for IMPORT INTO on a table that has columns typed as arrays of user-defined types (like enums). Tables that use multiple user-defined types with the same name but different schemas are still unsupported. #116359
• The new SELECT FOR UPDATE implementation used under Read Committed isolation (and under Serializable isolation when the optimizer_use_lock_op_for_serializable session variable is true) now locks all column families instead of only the first column family. #116826
• Fixed a bug where SELECT FOR UPDATE under Read Committed isolation on multi-column-family tables was not locking column families containing only key columns. #116826
• It is now possible to run CALL statements with EXPLAIN. The EXPLAIN (OPT) variant will show the body of the procedure, while other variants will only show the procedure name and arguments. #116274
• EXPLAIN output now contains detailed information about the plans for CASCADE actions. #117719

Operational changes

• Per-node hot ranges logging now logs the top 5 hot ranges on the local node instead of the top 5 hot ranges cluster-wide. #118334

Command-line changes

• Added a new command cockroach debug enterprise-check-fips, which diagnoses errors in FIPS deployments. #115202
• The new flag --enterprise-require-fips-ready can be added to any cockroach command to prevent startup if certain prerequisites for FIPS compliance are not met. #115202
• cockroach workload commands now appropriately invoke .Close in the case of an error. #116487

DB Console changes

• Updated the "CPU Time" label on the Runtime Dashboard to "SQL CPU Time" and added clarifications to its tooltip. #116449
• Statement bundles are now enabled for Serverless clusters. #117529
• The Networking Dashboard is enhanced with charts that visualize number of packets received, number of receiving packets with error, number of receiving packets that got dropped, number of packets sent, number of sending packets with error, and number of sending packets that got dropped. #116712
• The Explain Plans tab is now shown for the Statements and Insights pages, for Serverless clusters. #118169

Bug fixes

• Fixed a durability bug in Raft log storage, caused by incorrect syncing of filesystem metadata. Previously, it was possible to lose writes of a particular kind (AddSSTable) that were used by e.g. RESTORE. This loss was possible only under power-off or operating system crash conditions. Under such conditions, CockroachDB could enter a crash loop on node restart. In the worst case of a coordinated power-off/crash across multiple nodes this could lead to an unrecoverable loss of Raft quorum. #115709
• Fixed a bug where large jobs running with execution locality (such as some changefeeds) could result in the gateway node being assigned most of the work, causing performance degradation and cluster instability. #115388
• Fixed a bug that caused node crashes and panics when running INSERT queries on REGIONAL BY ROW tables with UNIQUE constraints or indexes. The bug was only present in v23.2.0-beta.1. #115668
• Fixed a bug that existed only in v23.2 alpha and beta versions that could have caused side effects to happen out of order for PL/pgSQL routines in rare cases. #115839
• Fixed a bug that existed since v23.1 that prevented naming user-defined type (UDT) parameters when dropping a user-defined function (or procedure). #115904
• Fixed a bug where scheduled jobs using external storage providers could fail shortly after node startup. #115693
• Locking tables (e.g., with SELECT FOR UPDATE) on the null-extended side of outer joins (e.g., the right side of a LEFT JOIN) is now disallowed and returns an error. This improves compatibility with PostgreSQL and prevents ambiguity in locking semantics. This bug has existed since locking with FOR UPDATE was introduced. #115878
• Fixed a display bug in the DB Console where because not all types of schema changes are setting the value for the mutation ID, the value of the ID could previously show as "with ID undefined" on the Events panel. Now, the notification omits the undefined value (the rest of the event notification is still displayed). #116518
• Fixed the formatting for PL/pgSQL routines, which could prevent creating a routine with loop labels, and could prevent some expressions from being redacted correctly. The bug only existed in v23.2 alpha and beta releases. #116713
• Fixed a bug that would cause a syntax error during redaction of a PL/pgSQL routine. The bug existed only in v23.2 alpha and beta releases. #116713
• Fixed a bug that would cause syntax errors when attempting to RESTORE a database with PL/pgSQL user-defined functions (UDFs) or stored procedures. This bug only affected v23.2 alpha and beta releases. #116713
• UPDATE, UPSERT, and INSERT ON CONFLICT queries are now disallowed under Read Committed isolation when the table contains a check constraint involving a column family that is updated, and the check constraint also involves a column family that is not updated, but is read. This is a temporary fix to prevent possible violation of the check constraint, and the restriction will be lifted in the future. #116428
• Previously, all AggHistogram-powered metrics were not reporting quantiles properly in the DB Console. This patch fixes the histograms so that the quantiles in DB Console are reported correctly. these histograms were only broken in the DB Console metrics dashboards, but were not broken in the Prometheus-compatible endpoint, /_status/vars. The list of affected metrics is shown below. #114506
  • changefeed.message_size_hist
  • changefeed.parallel_io_queue_nanos
  • changefeed.sink_batch_hist_nanos
  • changefeed.flush_hist_nanos
  • changefeed.commit_latency
  • changefeed.admit_latency
  • jobs.row_level_ttl.span_total_duration
  • jobs.row_level_ttl.select_duration
  • jobs.row_level_ttl.delete_duration
• Fixed a bug introduced in v23.2 that caused internal errors and panics when certain SQL queries were run with automatic index recommendation collection enabled. #117453
• Standard indexes and inverted indexes may no longer be created on PL/pgSQL REFCURSOR[]s columns. REFCURSOR columns themselves are not indexable. #116071
• Fixed a bug that prevented database RESTORE when the database contained a view or routine that referenced a user-defined type (UDT) in the body string. For views, this bug was introduced in v20.2, when user-defined types (UDTs) were introduced. For routines, this bug was introduced in v22.2, when user-defined functions (UDFs) were introduced. #116841
• Fixed a bug that could cause a function resolution error when attempting to use a builtin function like now() as a formatting argument to a PL/pgSQL RAISE statement. #116825
• Fixed a bug where CDC custom key columns did not function correctly with CDC queries. For example, CREATE CHANGEFEED WITH key_column=..., unordered AS SELECT * FROM table now works correctly instead of retrying forever. Note that some functionalities with CDC custom keys are not fully supported, see #115267 for more details. #116967
• Fixed a bug in Raft log truncation that could lead to crash loops, and unrecoverable loss of quorum in the unlikely worst case that all replicas enter this crash loop. The bug manifested when a few things coincided: The cluster was running a bulk write workload (e.g., schema change, import, RESTORE); a log truncation command was running; and the process crashed at an unfortunate moment (e.g., the process was killed, or killed itself for reasons like detecting a disk stall). #116574
• Fixed the value used for the total runtime on SQL statistics. This was using the wrong value previously, causing the SQL Activity page to display values with more than 100%. #117426
• Fixed a bug where trying to set an empty search_path session variable resulted in an error. #117557
• It is now possible to assign to the parameter of a PL/pgSQL routine. Previously, attempts to do this would result in a "variable not found" error at routine creation time. In addition, variable shadowing is now explicitly disabled, where previously it would cause an internal error. These bugs existed in the v23.2.0 release and the v23.2 pre-release versions. #117715
• Fixed a bug in the row-level TTL job that would cause it to skip expired rows if the primary key of the table included columns of the collated string type. This bug was present since the initial release of row-level TTL in v22.2.0. #117512
• Fixed a bug where concurrent GRANT statements can cause deadlocks. #117713
• CockroachDB can now transparently retry more retryable errors when performing a non-atomic COPY command. #117895
• Fixed a bug that caused DML statements to fail while a hash-sharded index was being created. The symptom of this bug was an error like column "crdb_internal_val_shard_16" does not exist. This bug was present since v23.1.0. #118215
• Previously, CockroachDB could encounter the error unable to encode table key: *tree.DTSQuery when operating on columns with the internal TSQuery type in some contexts (e.g., when collecting table statistics or when performing a DISTINCT operation). This is now fixed. The bug had been present since v23.1 when support for the internal TSQuery type was added. #118321
• Previously, in some cases CockroachDB could incorrectly evaluate queries that scanned an inverted index and had a WHERE filter in which two sides of the AND expression had "similar" expressions (e.g., ARRAY['str1'] <@ col AND (ARRAY['str1'] && col OR ...)); this is now fixed. The bug had been present since prior to v22.2. #118360
• Fixed a bug that could cause DELETE queries sent by the row-level TTL job to use a secondary index rather than the primary index to find the rows to delete. This could lead to some DELETE operations taking a much longer time than they should. This bug was present since v22.2.0. #118337
• Fixed an issue with missing data on SQL statistics, and consequently missing data on the SQL Activity page, by properly recalculating the value from the current and past hour on the top activity table. #118378
• Internal queries issued by the row-level TTL jobs should now use optimal plans. The bug has been present since at least v22.2. #118494
• Fixed a bug where a changefeed could omit events in rare cases, logging the error cdc ux violation: detected timestamp ... that is less or equal to the local frontier. This could happen in the following scenario:
  1. A rangefeed runs on a follower replica that lags significantly behind the leaseholder.
  2. A transaction commits and removes its transaction record before its intent resolution is applied on the follower.
  3. The follower's closed timestamp has advanced past the transaction commit timestamp.
  4. The rangefeed attempts to push the transaction to a new timestamp (at least 10 seconds after the transaction began).
  5. This may cause the rangefeed to prematurely emit a checkpoint before emitting writes at lower timestamps, which in turn may cause the changefeed to drop these events entirely, never emitting them. #118981

v23.2.0

Release Date: February 5, 2024

With the release of CockroachDB v23.2, we've added new capabilities to help you migrate, build, and operate more efficiently. See our summary of the most significant user-facing changes under Feature Highlights.

Downloads

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0.linux-arm64.tgz (SHA256)	cockroach-sql-v23.2.0.linux-arm64.tgz (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image, both Intel and ARM images are generally available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach:v23.2.0

Source tag

To view or download the source code for CockroachDB v23.2.0 on Github, visit v23.2.0 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-rc.2...v23.2.0

Feature highlights

This section summarizes the most significant user-facing changes in v23.2.0 and other features recently made available to CockroachDB users across versions. For a complete list of features and changes in v23.2, including bug fixes and performance improvements, see the release notes for previous v23.2 testing releases. You can also search the docs for sections labeled New in v23.2.

• Feature categories
  • Observability
  • Migrations
  • Security and compliance
  • Disaster recovery
  • Deployment and operations
  • SQL
• Additional information
  • Backward-incompatible changes
  • Deprecations
  • Known limitations
  • Additional resources

Note:

In CockroachDB Self-Hosted, all available features are free to use unless their description specifies that an Enterprise license is required. For more information, see the Licensing FAQ.

Observability

Feature	Availability
	Ver.	Self-Hosted	Dedicated	Serverless
Identify network partitions using updated metrics in the Network page The Network page in the DB console has been updated with additional metrics that surface when the cluster is unstable due to network partitions.	23.2
Troubleshoot 40001 errors from isolation conflicts in highly contended workloads Failed execution insights have been enhanced to include additional information (conflicting transaction and location) into 40001 errors that stem from isolation conflicts in highly contentious workloads.	23.2
Identify and troubleshoot your most recent transactions and statements in the console Users can control the refresh of SQL activity Active Executions. They can turn off automatic polling and instead manually refresh the page. The page will persist recent executions and their statistics and plans to allow users to troubleshoot their recent workload. Previously, the Active executions page would refresh automatically and cause potentially problematic executions to disappear from the page making it difficult to troubleshoot recent workloads.	23.2
Integrate SQL activity with Datadog for end-to-end observability SQL activity such as statements and transactions can be emitted with fine granularity to Datadog as logs to allow end to end observability using both metrics and logs.	23.2
Customize your own metric dashboard for CockroachDB serverless The CockroachDB Cloud console supports additional metrics that can be customized in a single dashboard for CockroachDB Serverless.	All*

Migrations

Feature	Availability
	Ver.	Self-Hosted	Dedicated	Serverless
Migrate to CockroachDB using Oracle Golden Gate You can now use Oracle Golden Gate to stream data directly into CockroachDB. Migrate data from compatible databases onto CockroachDB using this connector.	All**
Migrate to CockroachDB using Debezium You can now use Debezium to stream data directly into CockroachDB. Migrate data from compatible databases onto CockroachDB using this connector.	All**

Disaster recovery

Feature	Availability
	Ver.	Self-Hosted	Dedicated	Serverless
Physical Cluster Replication is now available in Preview Physical Cluster Replication is an asynchonous replication feature that allows your cluster to recover from full-cluster failure with a low RPO and RTO. In 23.2, it is a Preview feature, requiring an Enterprise license, and only available for self-hosted CockroachDB deployments.	23.2

Security and compliance

Feature	Availability
	Ver.	Self-Hosted	Dedicated	Serverless
Use Azure IAM to authenticate storage for changefeeds and backup/restore Use implicit authorization to leverage existing azure credentials to authenticate with your Azure Blob Storage bucket.	23.2
Folders UI in CockroachDB Cloud Orgs (Limited Access) CockroachDB Cloud organizations can now organize clusters using folders and can assign roles and permissions at the folder scope.	All*

Deployment and operations

Feature	Availability
	Ver.	Self-Hosted	Dedicated	Serverless
General Availability for ARM binaries for Linux and Docker With this release, Linux and Docker binaries for the ARM 64-bit architecture are Generally Available (GA) for production workloads. ARM binaries for macOS are **experimental** and not yet qualified for production use. For more details, installation instructions, and limitations, refer to Install CockroachDB on Linux.	23.2

SQL

Feature	Availability
	Ver.	Self-Hosted	Dedicated	Serverless
Support Read Committed Isolation Level in public preview Read Committed is a weaker transaction isolation level than Serializable and is the default isolation level in databases such as PostgreSQL, Oracle, and SQL Server. Read Committed isolation allows writes to interleave without aborting transactions and prevents writes from blocking reads, thus minimizing query latency and retries caused by read/write contention.	23.2
Improved UDF capabilities UDFs now support mutations (INSERT, UPDATE, UPSERT, and DELETE).	23.2
Support stored procedures and PL/pgSQL PL/pgSQL capabilities such as stored procedures, conditional logic, loops, and exception handling are now supported, increasing compatibility between CockroachDB and PostgreSQL. An Enterprise license is required.	23.2
Delete operation admission control This feature reduces the potential impact of deleting a large number of rows—directly, as well as implicitly via row-level TTL—on query performance and system stability. For more information, see Operations subject to admission control.	23.2
Replication admission control This mechanism automatically paces work for which longer exection times are acceptable (such as index backfills) at the speed of the slowest replica in order to maintain cluster stability and throughput. For more information, see Operations subject to admission control.	23.2
Column Level Encryption CockroachDB now supports column-level encryption through a set of built-in functions. This feature allows you to encrypt one or more columns in every row of a database table, and can be useful for compliance scenarios such as adhering to PCI or GDPR. An Enterprise license is required.	23.2
Revoke CREATE privilege by default for public schema Newer versions of PostgreSQL (15 and above) do not grant the CREATE privilege by default on the public schema. This change can be disruptive but to preserve Postgres compatibility it is now guarded behind a cluster setting as an opt in feature. This is controlled using a cluster setting: SET CLUSTER SETTING sql.auth.public_schema_create_privilege.enabled=true;	23.2
Easily find Default Privileges for a Role This feature adds a simple syntax to see the default privileges for a particular grantee. SHOW DEFAULT PRIVILEGES FOR GRANTEE root;	23.2

Feature detail key

*	Features marked “All*” were recently made available in the CockroachDB Cloud platform. They are available for all supported versions of CockroachDB, under the deployment methods specified in their row under Availability.
**	Features marked “All**” were recently made available via migration tools maintained outside of the CockroachDB binary. They are available to use with all supported versions of CockroachDB, under the deployment methods specified in their row under Availability.
	Feature is available for this deployment method of CockroachDB as specified in the icon’s column: CockroachDB Self-Hosted, CockroachDB Dedicated, or CockroachDB Serverless.
	Feature is not available for this deployment method of CockroachDB as specified in the icon’s column: CockroachDB Self-Hosted, CockroachDB Dedicated, or CockroachDB Serverless.

Backward-incompatible changes

Before upgrading to CockroachDB v23.2, be sure to review the following backward-incompatible changes, as well as key cluster setting changes, and adjust your deployment as necessary.

• The pre-v23.1 output produced by SHOW RANGES, crdb_internal.ranges, and crdb_internal.ranges_no_leases was deprecated in v23.1 and is now replaced by default with output that's compatible with coalesced ranges (anges that pack multiple tables/indexes/partitions into individual ranges). See the v23.1 release notes for SHOW RANGES for more details. #102961
• When a deployment is configured to use a time zone for log file output using formats crdb-v1 or crdb-v2, new output log entries cannot be processed by nodes that have not been upgraded to v23.2. #104265
• When customizing the SQL shell's interactive prompt, the special sequence %M now expands to the full host name instead of the combination of host name and port number. To include the port number explicitly, use %>. The special sequence %m now expands to the host name up to the first period. #105137
• The cockroach debug zip command stores data retrieved from SQL tables in the remote cluster using the TSV format by default. #107474
• The changefeed.protect_timestamp.max_age cluster setting will only apply to newly created changefeeds in v23.2. For existing changefeeds, you can set the protect_data_from_gc_on_pause option so that changefeeds do not experience infinite retries and accumulate protected change data. You can use the ALTER CHANGEFEED statement to add protect_data_from_gc_on_pause to existing changefeeds. #103539
• The direct export of traces to Jaeger and the cluster setting trace.jaeger.agent have been removed. The direct export functionality had been obsoleted since 2022; it stopped working altogether sometime in 2023 with the following error: data does not fit within one UDP packet; size 65006, max 65000, spans NN. Since 2022, Jaeger supports ingestion of traces using OTLP; and CockroachDB has supported emitting traces using OTLP since v22.1. Operators and developers who want to inspect traces are thus invited to use the OTLP protocol instead. The corresponding cluster setting is trace.opentelemetry.collector. For a successful deployment, an intermediate OTLP collector/forwarder should be configured. For an example of how to orchestrate the OpenTelemetry collector and Jaeger together using Docker Compose, or how to configure the otel-collector, see the more-detailed entry in v23.2-alpha.3 backward-incompatible changes. #111342

Key Cluster Setting Changes

The following changes should be reviewed prior to upgrading. Default cluster settings will be used unless you have manually set a value for a setting. This can be confirmed by checking the system.settings table (select * from system.settings) to view the non-default settings.

• The new cluster setting sql.txn.read_committed_syntax.enabled, controls whether transactions run under READ COMMITTED or SERIALIZABLE isolation. It defaults to false. When set to true, the following statements will configure transactions to run under READ COMMITTED isolation:

  • BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED
  • SET TRANSACTION ISOLATION LEVEL READ COMMITTED
  • SET default_transaction_isolation = 'read committed'
  • SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL READ COMMITTED

  #110624

• The sql.txn.read_committed_syntax.enabled cluster setting was renamed to sql.txn.read_committed_isolation.enabled. #113833

• Users who have the CREATEROLE role option can now grant and revoke role membership in any non-admin role. This change also removes the sql.auth.createrole_allows_grant_role_membership.enabled cluster setting, which was added in v23.1. In v23.2, the cluster setting is effectively always true. #104376

• The cluster setting sql.metrics.statement_details.gateway_node.enabled now defaults to false to reduce the number of rows generated in SQL Statistics pages. #107788

• The cluster setting kv.rangefeed.enabled no longer controls access to RANGEFEED SQL commands. Instead, use feature.changefeed.enabled. #110676

• The cluster settings related to physical cluster replication have been renamed for consistency. For example, bulkio.stream_ingestion.minimum_flush_interval is now named physical_replication.consumer.minimum_flush_interval. #111197

• CockroachDB now periodically dumps the state of its internal memory accounting system into the heap_profiler/ directory when a heap profile is taken. To disable this behavior, set the diagnostics.memory_monitoring_dumps.enabled cluster setting to false. #114998

• Introduced the cluster setting kv.gc.sticky_hint.enabled in v23.1.13. This setting helps expedite garbage collection after range deletions. For example, when a SQL table or index is dropped. kv.gc.sticky_hint.enabled is enabled by default and deprecated in v23.2. #113040

• CockroachDB now enables the pacing mechanism in rangefeed closed timestamp notifications, by setting the default kv.rangefeed.closed_timestamp_smear_interval cluster setting to 1ms. This makes rangefeed closed timestamp delivery more uniform and less spikey, which reduces its impact on the Go scheduler and, ultimately, foreground SQL latencies. #103006

Deprecations

• The protect_data_from_gc_on_pause option has been deprecated. This option is no longer needed since changefeed jobs always protect data. #103539
• The cockroach connect functionality has been deprecated. #114241

Known limitations

For information about new and unresolved limitations in CockroachDB v23.2, with suggested workarounds where applicable, see Known Limitations.

Additional resources

Resource	Topic	Description
Cockroach University	Introduction to Distributed SQL and CockroachDB	This course introduces the core concepts behind distributed SQL databases and describes how CockroachDB fits into this landscape. You will learn what differentiates CockroachDB from both legacy SQL and NoSQL databases and how CockroachDB ensures consistent transactions without sacrificing scale and resiliency. You'll learn about CockroachDB's seamless horizontal scalability, distributed transactions with strict ACID guarantees, and high availability and resilience.
Cockroach University	Practical First Steps with CockroachDB	This course will give you the tools you need to get started with CockroachDB. During the course, you will learn how to spin up a cluster, use the Admin UI to monitor cluster activity, and use SQL shell to solve a set of hands-on exercises.
Cockroach University	Enterprise Application Development with CockroachDB	This course is the first in a series designed to equip you with best practices for mastering application-level (client-side) transaction management in CockroachDB. We'll dive deep on common differences between CockroachDB and legacy SQL databases and help you sidestep challenges you might encounter when migrating to CockroachDB from Oracle, PostgreSQL, and MySQL.
Cockroach University	Building a Highly Resilient Multi-region Database using CockroachDB	This course is part of a series introducing solutions to running low-latency, highly resilient applications for data-intensive workloads on CockroachDB. In this course we focus on surviving large-scale infrastructure failures like losing an entire cloud region without losing data during recovery. We’ll show you how to use CockroachDB survival goals in a multi-region cluster to implement a highly resilient database that survives node or network failures across multiple regions with zero data loss.
Docs	Migration Overview	This page summarizes the steps of migrating a database to CockroachDB, which include testing and updating your schema to work with CockroachDB, moving your data into CockroachDB, and testing and updating your application.
Docs	Architecture Overview	This page provides a starting point for understanding the architecture and design choices that enable CockroachDB's scalability and consistency capabilities.
Docs	SQL Feature Support	The page summarizes the standard SQL features CockroachDB supports as well as common extensions to the standard.
Docs	Change Data Capture Overview	This page summarizes CockroachDB's data streaming capabilities. Change data capture (CDC) provides efficient, distributed, row-level changefeeds into a configurable sink for downstream processing such as reporting, caching, or full-text indexing.
Docs	Backup Architecture	This page describes the backup job workflow with a high-level overview, diagrams, and more details on each phase of the job.

v23.2.0-rc.2

Release Date: January 9, 2024

Downloads

Warning:

CockroachDB v23.2.0-rc.2 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-rc.2.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-rc.2.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-rc.2.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-rc.2.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-rc.2.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-rc.2.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-rc.2.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-rc.2.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-rc.2.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-rc.2.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is in Limited Access.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-rc.2

Source tag

To view or download the source code for CockroachDB v23.2.0-rc.2 on Github, visit v23.2.0-rc.2 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-rc.1...v23.2.0-rc.2

Bug fixes

• Fixed a bug introduced in v23.2 that caused internal errors and panics when certain queries ran with automatic index recommendation collection enabled. #117454
• Fixed a bug where mixed-version clusters with both v23.1 and v23.2 nodes could detect a false-positive replica inconsistency in GLOBAL tables. #117341

Contributors

This release includes 12 merged PRs by 9 authors.

v23.2.0-rc.1

Release Date: December 21, 2023

Downloads

Warning:

CockroachDB v23.2.0-rc.1 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-rc.1.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-rc.1.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-rc.1.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-rc.1.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-rc.1.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-rc.1.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-rc.1.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-rc.1.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-rc.1.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-rc.1.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is in Limited Access.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-rc.1

Source tag

To view or download the source code for CockroachDB v23.2.0-rc.1 on Github, visit v23.2.0-rc.1 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-beta.3...v23.2.0-rc.1

Enterprise edition changes

• Added a SQL function crdb_internal.fips_ready() that can be used to verify the FIPS readiness of the gateway node. #116281
• Physical cluster replication now retries for just over 3 minutes before failing. #116404

SQL language changes

• CALL statements can now be run with EXPLAIN. The EXPLAIN (OPT) variant will show the body of the procedure, while other variants will show only the procedure name and arguments. #116273
• Added support for IMPORT INTO a table that has columns typed as arrays of user-defined types (like ENUM). Tables that use multiple user-defined types with the same name but different schemas are still unsupported. #116360
• The SELECT FOR UPDATE implementation used under Read Committed isolation (and under Serializable isolation when optimizer_use_lock_op_for_serializable is set to true) now locks all column families instead of only the first column family. #116828

Command-line changes

• Added the command cockroach debug enterprise-check-fips that diagnoses errors in FIPS deployments. #116281
• Added the flag --enterprise-require-fips-ready that can be run with any CockroachDB command to prevent startup if certain prerequisites for FIPS compliance are not met. #116281

DB Console changes

• Updated the CPU Time label to SQL CPU Time and added clarification to its tooltip on the SQL Activity and Insights pages. #116450
• Removed the ID when it is undefined from the event description in the Metrics Events Panel. #116519

Bug fixes

• Fixed a bug that caused node crashes and panics when running INSERT queries on REGIONAL BY ROW tables with UNIQUE constraints or indexes. The bug is only present in version v23.2.0-beta.1. #116343
• UPDATE, UPSERT, and INSERT ON CONFLICT queries are now disallowed under Read Committed isolation when the table contains a CHECK constraint involving a column family that is updated, and that CHECK constraint also involves a column family that is not updated, but is read. This restriction is a temporary fix to prevent possible violation of the CHECK constraint. However, it is important to note that this restriction will be lifted in the future. #116429
• Fixed a bug where scheduled jobs using external storage providers may fail shortly after node startup. #116205
• Fixed the formatting for plpgsql routines, which could prevent the creation of a routine with loop labels and could prevent some expressions from being redacted correctly. The bug only existed in alpha and beta versions of v23.2. #116711
• Fixed a bug that would cause a syntax error during redaction of a PL/pgSQL routine. The bug existed only in alpha and beta versions of the v23.2 release. #116711
• Fixed a bug that would cause syntax errors when attempting to restore a database with PL/pgSQL UDFs or stored procedures. This bug only affected alpha and beta versions of v23.2. #116711
• Fixed a bug in PL/pgSQL where altering the name of a sequence or UDT that was used in a PL/pgSQL function or procedure could break them. This is only present in v23.2 alpha and beta releases. #116420
• Fixed a bug where SELECT FOR UPDATE under Read Committed isolation on multi-column-family tables was not locking column families containing only key columns. #116828

• Fixed a bug where all AggHistogram-powered metrics were not reporting quantiles properly in the DB Console. The quantiles in the DB Console are now reported correctly. This bug was only present in histograms in the DB Console metrics features, and did not affect metrics reporting in the Prometheus-compatible endpoint, /_status/vars. The affected metrics were:

  • changefeed.message_size_hist
  • changefeed.parallel_io_queue_nanos
  • changefeed.sink_batch_hist_nanos
  • changefeed.flush_hist_nanos
  • changefeed.commit_latency
  • changefeed.admit_latency
  • jobs.row_level_ttl.span_total_duration
  • jobs.row_level_ttl.select_duration
  • jobs.row_level_ttl.delete_duration

  #116871

Contributors

This release includes 49 merged PRs by 26 authors.

v23.2.0-beta.3

Release Date: December 13, 2023

Downloads

Warning:

CockroachDB v23.2.0-beta.3 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-beta.3.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-beta.3.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-beta.3.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-beta.3.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-beta.3.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-beta.3.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-beta.3.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-beta.3.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-beta.3.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-beta.3.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is Experimental and not yet qualified for production use and not eligible for support or uptime SLA commitments.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-beta.3

Source tag

To view or download the source code for CockroachDB v23.2.0-beta.3 on Github, visit v23.2.0-beta.3 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-beta.2...v23.2.0-beta.3

General changes

• Updated Go version to 1.21.3. #116098

SQL language changes

• Added the sql.ttl.default_select_rate_limit cluster setting and the ttl_select_rate_limit table storage parameter to set the TTL select rate limit. This sets the number of records per table per second per node that can be selected by the TTL job. #115802

Bug fixes

• Fixed a bug that could result in an incorrect too few columns error for queries that use ANY <array> syntax with a subquery. #115592
• Fixed a bug that could cause too few columns/too many columns errors for queries that used IN or NOT IN with a non-trivial right operand, such as a subquery (rather than a constant tuple). #115592
• Fixed a bug where CREATE INDEX with expressions could fail on materialized views when the declarative schema changer was used. #115522
• Fixed a bug that could cause PL/pgSQL routines with SELECT INTO syntax to return early. This bug existed only in pre-release versions v23.2.0-beta.1 and v23.2.0-beta.2. #115676
• Fixed a bug that could cause side effects to happen out of order for PL/pgSQL routines in rare cases. This bug existed only in v23.2 alpha versions and previous v23.2 beta versions. #115840
• Previously, in rare cases, CockroachDB could incorrectly evaluate queries with lookup joins where equality cols are key when performing lookups on multiple ranges. This could either manifest as a stuck query or result in incorrect output. The bug was introduced in v22.2 and is now fixed. #115580
• Fixed a durability bug in Raft log storage that was caused by incorrect syncing of filesystem metadata. It was possible to lose writes of a particular kind (AddSSTable) used by (e.g.) RESTORE. This loss was possible only under power-off or OS crash conditions. As a result, CockroachDB could enter a crash loop on restart. In the worst case of a coordinated power-off/crash across multiple nodes, this could lead to an unrecoverable loss of quorum. #115841
• Fixed a bug where large jobs running with execution locality option could result in the gateway node being assigned most of the work causing performance degradation and cluster instability. #115876
• Fixed a bug that prevented naming UDT parameters when dropping a user-defined function (or procedure). This bug has existed since v23.1. #115905
• Locking tables (e.g., with SELECT ... FOR UPDATE) on the null-extended side of outer joins (e.g., the right side of a LEFT JOIN) is now disallowed and returns an error. This improves compatibility with PostgreSQL and prevents ambiguity in locking semantics. This bug has existed since locking with FOR UPDATE was introduced. #115879

Contributors

This release includes 26 merged PRs by 20 authors.

v23.2.0-beta.2

Release Date: December 5, 2023

Downloads

Warning:

CockroachDB v23.2.0-beta.2 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-beta.2.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-beta.2.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-beta.2.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-beta.2.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-beta.2.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-beta.2.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-beta.2.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-beta.2.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-beta.2.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-beta.2.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is Experimental and not yet qualified for production use and not eligible for support or uptime SLA commitments.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-beta.2

Source tag

To view or download the source code for CockroachDB v23.2.0-beta.2 on Github, visit v23.2.0-beta.2 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-beta.1...v23.2.0-beta.2

General changes

• CockroachDB now periodically dumps the state of its internal memory accounting system into the heap_profiler/ directory when a heap profile is taken. To disable this behavior, set the diagnostics.memory_monitoring_dumps.enabled cluster setting to false. #114998
• Multi-level compactions have been disabled to investigate possible performance issues with foreground throughput and latency. #115481

Enterprise Edition changes

• When using Physical Cluster Replication, you can now initiate a cutover as of LATEST before the initial scan completes. #115101
• Sensitive information such as api_secret, sasl_password, client_cert, and ca_cert, is now redacted in output from commands SHOW CHANGEFEED JOB, SHOW CHANGEFEED JOBS, and SHOW JOBS. #115567
• The physical_replication.frontier_lag_nanos metric and the related DB Console graph have been removed because they sometimes display incorrect information. For alerting, it is recommended to use the new metric physical_replication.replicated_time_seconds metric instead. #115234
• Fixed a bug in physical cluster replication where replicating from a primary cluster that is on a version prior to v23.2.x to a standby cluster running on v23.2.x could fail because of an undefined builtin function in the primary cluster. #114257

DB Console changes

• In the Changeeds dashboard, the Max Checkpoint Latency chart title now refers to "Lag" rather than "Latency", to better reflect the intention of the underlying metric, which measures how recently the changefeed was last checkpointed. #115003
• Times on the X-Axis of bar charts in Statement details pages are now correctly formatted in UTC. #115220
• In the SQL Activity Transaction Details page, you can now view a transaction fingerprint ID across multiple applications by specifying the application name in the appNames URL GET parameter using a comma-separated encoded string of transaction fingerprint IDs. #115204

Bug fixes

• Fixed a bug that prevented the Now button on time range selectors in the DB Console from working as expected when a custom time period was previously selected. #115514
• Fixed a bug that prevented the SQL Activity page from showing internal statements when the sql.stats.response.show_internal.enabled cluster setting was set to true. #114824
• Fixed a bug where an active replication report update could get stuck in a retry loop on clusters with over 10000 ranges. This could prevent a node from shutting down cleanly. #114178
• Fixed a bug introduced in v23.1 that could cause an internal error when using the text format (as opposed to binary) when preparing a statement with a user-defined composite type. #115064
• Fixed a bug that could cause a replica to be stuck processing in a queue's replica set when the replica had recently been removed from purgatory for processing but was destroyed, or the replica's ID changed before being processed. These replicas are now removed from the queue when they are encountered. #115037
• Fixed a bug that could cause a prepared statement to fail if it references both an enum and a table that has undergone a schema change. #115132
• Fixed a bug that could cause cluster version finalization to contend with descriptor lease renewals on large clusters. Descriptor lease renewals previously had a higher priority than cluster upgrade finalization. Finalization now always has a higher priority than descriptor lease renewal. #115034
• Fixed a bug that prevented backups from distributing work evenly across all replicas, including followers, regardless of leaseholder placement. #115019
• Fixed a bug introduced in v23.2.0-beta.1 that could cause a single composite-typed variable to be incorrectly handled as the target of a PostgreSQL INTO clause. #115404
• Fixed a bug that could cause a BEGIN statement log to record incorrect information in the Age field, which could also cause them to appear erroneously in slow-query logs. #115259

Performance improvements

• Query planning time has been reduced significantly for some queries in which many tables are joined. #114445

Contributors

This release includes 91 merged PRs by 35 authors.

v23.2.0-beta.1

Release Date: November 27, 2023

Downloads

Warning:

CockroachDB v23.2.0-beta.1 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-beta.1.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-beta.1.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-beta.1.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-beta.1.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-beta.1.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-beta.1.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-beta.1.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-beta.1.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-beta.1.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-beta.1.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is Experimental and not yet qualified for production use and not eligible for support or uptime SLA commitments.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-beta.1

Source tag

To view or download the source code for CockroachDB v23.2.0-beta.1 on Github, visit v23.2.0-beta.1 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-alpha.7...v23.2.0-beta.1

SQL language changes

• COPY commands now use the background quality-of-service level by default, which makes COPY commands subject to admission control. The new session variable copy_transaction_quality_of_service controls the quality-of-service level for COPY commands. Previously, COPY used the same level as other commands, determined by the default_transaction_quality_of_service session variable, which is set to regular by default. regular is not subject to admission control. #114535

DB Console changes

• The Overview page now correctly renders the background color for the email signup, which fixes an issue where it was difficult to read the text. #114547
• Fixed a bug where selecting the internal application name prefix $ internal from the Application Name dropdown on the SQL Activity Statements page was not showing internal queries. The filtering logic will now show if there are statements with the $ internal application name prefix. #114517

Bug fixes

• Fixed a bug where an empty range corresponding to a DROP TABLE did not respect system-level span configurations such as protected timestamps, which potentially caused reads above the protected timestamp to fail. #114833
• Fixed error handling for GetFiles so that it does not cause a nil pointer dereference. #114830

Contributors

This release includes 33 merged PRs by 21 authors.

v23.2.0-alpha.7

Release Date: November 20, 2023

Downloads

Warning:

CockroachDB v23.2.0-alpha.7 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-alpha.7.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.7.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.7.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-alpha.7.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-alpha.7.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.7.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.7.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.7.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-alpha.7.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-alpha.7.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is Experimental and not yet qualified for production use and not eligible for support or uptime SLA commitments.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-alpha.7

Source tag

To view or download the source code for CockroachDB v23.2.0-alpha.7 on Github, visit v23.2.0-alpha.7 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-alpha.6...v23.2.0-alpha.7

SQL language changes

• Previously, if session variable use_declarative_schema_changer was set to off, then ALTER TABLE ... ALTER COLUMN ... SET NOT NULL was run on a column which contained a NULL value, an error with code 23514 (check_violation) would be returned. Now in this scenario the error returned will have code 23502 (not_null_violation) to match PostgreSQL. #113970
• The sql.txn.read_committed_syntax.enabled cluster setting was renamed to sql.txn.read_committed_isolation.enabled. #113833

Command-line changes

• The cockroach connect functionality has been deprecated. #114241

DB Console changes

• Previously, the forward arrow button on the time selector would not move the time window forward if the current end time was less than "Now() - time window". For example, with a 10 minute time window, it was not possible to move forward if current end time is less that "Now() - 10 minutes". This caused the forward arrow button to become disabled even though there was more data to display. Now this scenario is handled by the forward arrow button selecting the latest available time window (similar to the Now button). #113907

Bug fixes

• Removed duplication of metrics names on DB Console Metrics charts' tooltips. #113728
• Fixed a bug that could cause ALTER DATABASE ... ADD/DROP REGION to hang if node localities were changed after regions were added. #114102
• A bug in the log configuration code prevented users from setting the datetime-format and datetime-timezone log format options (set via the format-options structure) within their log configuration. Specifically, when users tried to use these options in file-defaults with any json type log format, the log configuration was previously unable to be parsed due to validation errors. This was because the file-defaults.format-options were propagated to the sinks.stderr.format-options. sinks.stderr only supports a format of crdb-v2-tty. Therefore, the incorrectly propagated format-options, which are only supported by the json log format, were identified as not being supported when validating sinks.stderr. This bug is now fixed and the file-defaults.format-options are only propagated to sinks.stderr.format-options if both of these conditions are true: 1. file-defaults.format is one of crdb-v2 or crdb-v2-tty. 2. sinks.stderr.format-options are not explicitly set in the log configuration. #113684
• Previously, when executing queries with index joins or lookup joins or both when the ordering needs to be maintained, CockroachDB in some cases would get into a pathological behavior which would lead to increased query latency, possibly by one or two orders of magnitude. This bug was introduced in v22.2 and is now fixed. #114117
• Previously, the SHOW STATISTICS command incorrectly required the user to have the admin role. Now, it correctly only requires the user to have any privilege on the table being inspected. #114449
• Fixed a bug that could cause a query plan to skip scanning rows from the local region when performing a lookup join with a REGIONAL BY ROW table as the input. #114458

Performance improvements

• This change prevents failed requests from being issued on follower nodes that are draining, decommissioning or unhealthy which prevents latency spikes if those nodes later go offline. #114259

Contributors

This release includes 95 merged PRs by 33 authors.

v23.2.0-alpha.6

Release Date: November 7, 2023

Downloads

Warning:

CockroachDB v23.2.0-alpha.6 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-alpha.6.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.6.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.6.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-alpha.6.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-alpha.6.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.6.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.6.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.6.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-alpha.6.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-alpha.6.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is Experimental and not yet qualified for production use and not eligible for support or uptime SLA commitments.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-alpha.6

Source tag

To view or download the source code for CockroachDB v23.2.0-alpha.6 on Github, visit v23.2.0-alpha.6 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-alpha.5...v23.2.0-alpha.6

General changes

• The CockroachDB Docker image is now based on Red Hat's ubi9/ubi-minimal image instead of the ubi8/ubi-minimal image. #112967

SQL language changes

• Added the built-in function jsonb_array_to_string_array that converts JSONB array to STRING array. #112865
• The built-in function jsonb_array_to_string_array can now return NULL objects. #112865

Operational changes

• Introduced the cluster setting kv.gc.sticky_hint.enabled that helps expediting garbage collection after range deletions. For example, when a SQL table or index is dropped. kv.gc.sticky_hint.enabled is enabled by default in v23.2. The setting has been deprecated in v23.2. #113040
• Introduced a new environment variable that allows an operator to configure the compaction concurrency. #113313
• Debug zip will now collect the active traces of all running or reverting traceable jobs. This includes restores, imports, backups, and physical cluster replication. #113172

Cluster virtualization

• The privilege that controls access to CREATE VIRTUAL CLUSTER and other virtual cluster management syntax is now called MANAGEVIRTUALCLUSTER. #113076

Bug fixes

• Fixed a bug that could prevent RESTORE from working if it was performed during a cluster upgrade. #112759
• Fixed a bug where the opclass for a trigram index is not shown if CockroachDB creates a trigram index and later displays it via SHOW CREATE TABLE. #113071
• Fixed a bug where CockroachDB could incorrectly evaluate lookup and index joins into tables with at least three column families. This would result in either the non-nullable column with no value internal error, or the query would return incorrect results. This bug was introduced in v22.2. #113105
• Fixed a bug where ALTER PRIMARY KEY would incorrectly disable secondary indexes while new secondary indexes were being backfilled when using the declarative schema changer. #112627
• Fixed a bug where the unique_constraint_catalog and unique_constraint_schema columns in information_schema.referential_constraints could be incorrect for cross schema or cross database references. #112739
• Fixed a bug in a method that was used by some of the jobs observability infrastructure. This method could be triggered if a file was overwritten with a different chunking strategy. #113290
• Fixed a bug where the result of SHOW CREATE TABLE for a table that had a collated string column with a default expression was incorrect because the statement could not be parsed. #113119
• Fixed the SQL activity update job to: avoid conflicts on update, reduce the amount of data cached to only what the overview page requires, and fix the correctness of the top queries. #112865
• Fixed a bug that could prevent physical cluster replication from advancing in the face of some range deletion operations. #113041
• Fixed a bug where ALTER TYPE could get stuck if DROP TYPE was executed concurrently. #113644
• Fixed a bug that could cause internal errors or panics while attempting to forecast statistics on a numeric column. #113797
• Rolled back deletes no longer cause a discrepancy between computed statistics and the actual stored values. #113766

Performance improvements

• Addressed a performance regression that can happen when the declarative schema changer is used to create an index with a concurrent workload. #113725

Contributors

This release includes 117 merged PRs by 49 authors.

v23.2.0-alpha.5

Release Date: October 30, 2023

Downloads

Warning:

CockroachDB v23.2.0-alpha.5 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-alpha.5.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.5.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.5.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-alpha.5.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-alpha.5.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.5.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.5.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.5.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-alpha.5.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-alpha.5.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is Experimental and not yet qualified for production use and not eligible for support or uptime SLA commitments.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-alpha.5

Source tag

To view or download the source code for CockroachDB v23.2.0-alpha.5 on Github, visit v23.2.0-alpha.5 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-alpha.4...v23.2.0-alpha.5

SQL language changes

• Added support for the special OTHERS condition in PL/pgSQL exception blocks, which allows matching any error code apart from query_canceled and assert_failure. Note that Class 40 errors (40000, 40001, 40003, 40002, and 40P01) cannot be caught either. This is tracked in #111446. #112817

Bug fixes

• Previously, queries with the ST_Union aggregate function could produce incorrect results in some cases due to the query optimizer performing invalid optimizations. This is now fixed. This bug had been present since the ST_Union function was introduced in v20.2.0. #112780

Contributors

This release includes 27 merged PRs by 17 authors.

v23.2.0-alpha.4

Release Date: October 23, 2023

Downloads

Warning:

CockroachDB v23.2.0-alpha.4 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-alpha.4.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.4.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.4.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-alpha.4.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-alpha.4.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.4.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.4.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.4.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-alpha.4.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-alpha.4.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is Experimental and not yet qualified for production use and not eligible for support or uptime SLA commitments.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-alpha.4

Source tag

To view or download the source code for CockroachDB v23.2.0-alpha.4 on Github, visit v23.2.0-alpha.4 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-alpha.3...v23.2.0-alpha.4

General changes

• Updated the licenses/CCT.txt file to reflect the latest Cockroachdb Community License. #112494

Enterprise edition changes

• Renamed cluster settings related to physical cluster replication for consistency. For example, bulkio.stream_ingestion.minimum_flush_interval is now physical_replication.consumer.minimum_flush_interval. #111197

SQL language changes

• SHOW SCHEDULES has two columns that surface the schedule options. These columns have been renamed to align with the documented option names: on_previous_running and on_execution_failure. #111759
• Added support for the PLpgSQL CLOSE statement, which allows a PLpgSQL routine to close a cursor with the name specified by a cursor variable. #111330
• When a RESTORE with remove_regions is performed, the restore job will now fail if the object contains a REGIONAL BY ROW table. #111443
• It is now possible to open a cursor within a PLpgSQL function or procedure with an exception block. If an error occurs, creation of the cursor is rolled back before control reaches the exception handler. #111735
• If a scheduled backup resumes on a new cluster (e.g., after physical cluster replication cutover or a cluster restore), the backup schedule will pause. The user may resume the schedule without changing it, but should take special care to ensure no other schedule is backing up to the same collection. The user may also want to cancel the paused schedule and start a new one. #111578
• Added support for PLpgSQL FETCH and MOVE statements. Similar to SQL FETCH/MOVE statements, commands that would seek the cursor backward will fail. In addition, expressions other than constant integers are not yet supported for the count option. #111318
• Added support for the REFCURSOR data type. REFCURSOR is a special string type that is used to handle cursors. PLpgSQL cursor declarations are required to use a variable of type REFCURSOR, and the name of a cursor can be passed to and from a PLpgSQL function or procedure. #111392

• Added two changes to FOR UPDATE:

  • Multiple FOR UPDATE clauses on fully parenthesized queries are now disallowed. For example, the following statements are now disallowed:

    (SELECT 1 FOR UPDATE) FOR UPDATE;
    SELECT * FROM ((SELECT 1 FOR UPDATE) FOR UPDATE) AS x;
    

    Whereas statements like the following are still allowed:

    SELECT * FROM (SELECT 1 FOR UPDATE) AS x FOR UPDATE;
    SELECT (SELECT 1 FOR UPDATE) FOR UPDATE;
    

    This does not match PostgreSQL, which allows all of these, but does match CockroachDB behavior for ORDER BY and LIMIT.

  • FOR UPDATE is now allowed on statements with VALUES in the FROM list, or as a subquery. For example, the following statements are now allowed:

    SELECT (VALUES (1)) FOR UPDATE;
    SELECT * FROM (VALUES (1)) AS x FOR UPDATE;
    

    Using FOR UPDATE directly on VALUES is still disallowed:

    VALUES (1) FOR UPDATE; (VALUES (1)) FOR UPDATE;
    INSERT INTO t VALUES (1) FOR UPDATE;
    

    This matches PostgreSQL. #111258

• FOR UPDATE is now permitted on some queries that were previously disallowed. Queries that use the following operations are now allowed to have FOR UPDATE OF as long as the prohibited operation is in a subquery not locked by the FOR UPDATE OF:

  • UNION
  • INTERSECT
  • EXCEPT
  • DISTINCT
  • GROUP BY
  • HAVING
  • Aggregations
  • Window functions

  For example, the following query is now allowed because the subquery using the prohibited operations is not affected by the FOR UPDATE OF:

  SELECT * FROM t, (SELECT DISTINCT 0, 0 UNION SELECT a, count(*) FROM t GROUP BY a HAVING a > 0) AS u FOR UPDATE OF t;
  

  This matches PostgreSQL. #111258

• Identifiers after numeric constants that are not separated by whitespace are now disallowed to match PostgreSQL 15 behavior. #112021

• Added the new column contention_type to the crdb_internal.transaction_contention_events table. This column indicates the type of transaction contention encountered. Current values are LOCK_WAIT and SERIALIZATION_CONFLICT. #111685

• Changed the error message: statement error cannot execute FOR UPDATE in a read-only transaction to statement error cannot execute SELECT FOR UPDATE in a read-only transaction to match PostgreSQL. #112138

• Added a new session variable optimizer_use_lock_op_for_serializable, which when set enables a new implementation of SELECT FOR UPDATE. This new implementation of SELECT FOR UPDATE acquires row locks after any joins and filtering, and always acquires row locks on the primary index of the table being locked. This more closely matches SELECT FOR UPDATE behavior in PostgreSQL, but at the cost of more round trips from gateway node to replica leaseholder. Under read-committed isolation (and other isolation levels weaker than serializable), CockroachDB will always use this new implementation of SELECT FOR UPDATE regardless of the value of optimizer_use_lock_op_for_serializable to ensure correctness. #112138

Operational changes

• Added a new cluster setting server.http.base_path that controls the redirection of the browser after successful login with OIDC SSO. It is unlikely that this setting would need adjustment. However, it is helpful in cases where CockroachDB is running behind a load balancer or proxy that serves CockroachDB under a subpath, such as https:// <hostname>/crdb/. In those cases, it is necessary for the browser to redirect to / crdb after login instead of /, which has always been the hard-coded default. #111283

Cluster virtualization

• The following settings can now only be set from the system virtual cluster:

  • All the physical_replication.* settings
  • server.rangelog.ttl
  • timeseries.storage.*

  #111769

• The cluster settings cluster.organization and enterprise.license can now only be set via the system virtual cluster. Attempting to set them from any other virtual cluster results in an error. #111788

• A new flag --internal-rpc-port-range allows operators to specify the port range used by virtual clusters for node-to-node communication. Users implementing physical cluster replication or cluster virtualization public preview features should use this flag if they require the cockroach processes to only communicate using ports in a known port range. #111798

• Two guardrails are available to system operators to help with users upgrading from a deployment without cluster virtualization enabled to a deployment using cluster virtualization. This is intended to help in cases where the user is not connected to the correct SQL interface to perform certain configuration operations. There are two guardrails included:

  • The sql.restrict_system_interface.enabled cluster setting encourages users to use a virtual cluster for their application workload. When set, certain common operations that end users may execute to set up an application workload are disallowed, such as running DDL statements or modifying an application level cluster setting. Users will receive an error similar to:

    ERROR: blocked DDL from the system interface SQLSTATE: 42501 HINT: Object creation blocked via sql.restrict_system_interface.enabled to prevent likely user errors. Try running the DDL from a virtual cluster instead.
    

  • The sql.error_tip_system_interface.enabled cluster setting enhances errors reported when a user mistakenly uses a storage-level SQL feature within any virtual cluster besides the system virtual cluster. For example, when attempting to modify a cluster setting that was previously at the application level, an error like the following occurs:

    NOTICE: ignoring attempt to modify "kv.rangefeed.enabled"
    HINT: The setting is only modifiable by the operator.
    Normally, an error would be reported, but the operation is silently accepted here as configured by "sql.error_tip_system_interface.enabled".
    

  • For a cluster setting that was always system-level, an error like the following occurs:

    ERROR: cannot modify storage-level setting from virtual cluster
    SQLSTATE: 42501
    HINT: Connect to the system interface and modify the cluster setting from there.
    

    #111568

• The predefined config profiles related to cluster virtualization now automatically set the new cluster settings sql.restrict_system_interface.enabled and sql.error_tip_system_interface.enabled. #111568

• The hidden --secondary-tenant-port-offset option has been removed. Users who were previously using this option should use --internal-rpc-port-range instead. #112050

• Added support for automatic finalization of a virtual cluster's version upgrade. A new setting cluster.auto_upgrade.enabled was added to enable and disable automatic cluster version upgrade (finalization). It will be used in automatic upgrade of both the storage cluster and its virtual clusters. #102427

Command-line changes

• cockroach debug zip has an additional flag that is default off include-running-job-traces that will enable collecting the in-flight traces of traceable jobs, such as backup, restore, import, physical cluster replication and dump them in a jobs/ subdirectory in the zip. #112644

DB Console changes

• The Jobs table will now correctly display timestamps for creation, last modified, and the completed time fields. #110366
• The transaction insight details will show the following details when CockroachDB has information on a transaction execution with a 40001 error code and it has captured the conflicting transaction meta details (only available if the transaction had not yet committed at the time of execution). A section called Failed Execution will appear when this information is available and it will contain:
  • Blocking transaction execution ID
  • Blocking transaction fingerprint ID
  • Conflict location
  • Database, table, and index names #111873
• Added progressive loading functionality to the Databases page. #110901

Bug fixes

• Fixed a bug in physical cluster replication where the primary cluster would not be able to take backups when a primary cluster node was unavailable. #111337
• Fixed a bug in transaction insight details where it was possible to see the contention details of other transactions. Now, CockroachDB will only surface contention details for the current transaction. #111867
• Voter constraints will now be satisfied by promoting existing non-voters. Previously, there was a bug where voter constraints were never satisfied due to all existing replicas being considered necessary to satisfy a replica constraint. #111609
• Fixed a bug where indoption inside pg_index was not properly encoded causing clients to be unable to decode it as int2vector. #111911
• This patch fixes an issue where the optimizer fails to honor the statement_timeout session setting when generating constrained index scans for queries with large IN lists or = ANY predicates on multiple index key columns, which may lead to an out of memory condition on the node. #111979
• This patch fixes a performance issue in join queries with a LIMIT clause, where the optimizer may fail to push a WHERE clause filter into a join due to how the LIMIT operation is internally rewritten. This causes a full scan of the table referenced in the filter. #110593
• Fixed a bug that caused internal errors during query optimization in rare cases. The bug has been present since version v2.1.11, but it is more likely to occur in version v21.2.0 and later, though it is still rare. The bug only presents when a query contains min and max aggregate functions. #112255

Performance improvements

• This patch adds support for insert fast-path uniqueness checks on REGIONAL BY ROW tables where the source is a VALUES clause with a single row. This results in a reduction in latency for single-row inserts to REGIONAL BY ROW tables and hash-sharded REGIONAL BY ROW tables with unique indexes. #111822

Contributors

This release includes 213 merged PRs by 51 authors. We would like to thank the following contributors from the CockroachDB community:

• Finn Mattis (first-time contributor)
• craig

v23.2.0-alpha.3

Release Date: October 10, 2023

Downloads

Warning:

CockroachDB v23.2.0-alpha.3 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-alpha.3.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.3.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.3.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-alpha.3.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-alpha.3.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.3.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.3.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.3.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-alpha.3.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-alpha.3.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is Experimental and not yet qualified for production use and not eligible for support or uptime SLA commitments.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-alpha.3

Source tag

To view or download the source code for CockroachDB v23.2.0-alpha.3 on Github, visit v23.2.0-alpha.3 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-alpha.2...v23.2.0-alpha.3

Backward-incompatible changes

• The direct export of traces to Jaeger and the cluster setting trace.jaeger.agent have been removed. The direct export functionality had been obsoleted since 2022; it stopped working altogether sometime in 2023 with the following error: data does not fit within one UDP packet; size 65006, max 65000, spans NN. Since 2022, Jaeger supports ingestion of traces using OTLP; and CockroachDB has supported emitting traces using OTLP since v22.1. Operators and developers who want to inspect traces are thus invited to use the OTLP protocol instead. The corresponding cluster setting is trace.opentelemetry.collector. For a successful deployment, an intermediate OTLP collector/forwarder should be configured.

  • You can orchestrate the OpenTeletry collector and Jaeger together using Docker Compose by adapting the following example:

  otel-collector:
    image: otel/opentelemetry-collector-contrib
    container_name: otel-collector
    volumes:
      - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
    ports:
      - 1888:1888 # pprof extension
      - 8888:8888 # Prometheus metrics exposed by the collector
      - 8889:8889 # Prometheus exporter metrics
      - 13133:13133 # health_check extension
      - 4317:4317 # OTLP gRPC receiver
      - 4318:4318 # OTLP http receiver
      - 55679:55679 # zpages extension
  
  jaeger:
    image: jaegertracing/all-in-one
    container_name: jaeger
    ports:
      - "16685:16685"
      - "16686:16686"
      - "14250:14250"
      - "14268:14268"
      - "14269:14269"
      - "6831:6831/udp"
    environment:
      - COLLECTOR_ZIPKIN_HTTP_PORT=9411
      - COLLECTOR_OTLP_ENABLED=true
  

  • To configure the otel-collector, you can adapt this example:

  receivers:
    otlp: # the OTLP receiver the app is sending traces to
      protocols:
        grpc:
        http:
  
  processors:
    batch:
  
  exporters:
    otlp/jaeger: # Jaeger supports OTLP directly
      endpoint: http://jaeger:4317
      tls:
        insecure: true
  
  service:
    pipelines:
      traces/dev:
        receivers: [otlp]
        processors: [batch]
        exporters: [otlp/jaeger]
  

To use this configuration, unset Jaeger via SET CLUSTER SETTING trace.jaeger.agent='', and then set the OTLP collector using SET CLUSTER SETTING trace.opentelemetry.collector='localhost:4317'.

Enterprise edition changes

• Changefeeds now support the confluent-cloud:// sink scheme. This scheme can be used to connect to Kafka hosted on Confluent Cloud. The scheme functions identically to Kafka, but it has it's own authentication parameters. Namely, it requires api_key and api_secret to be passed as parameters in the sink URI. They must be URL encoded. An example URI is: 'confluent-cloud://pkc-lzvrd.us-west4.gcp.confluent.cloud:9092?api_key=<KEY>&api_secret=<SECRET>'. By default, the options tls_enabled=true, sasl_handshake=true, sasl_enabled=true, and sasl_mechanism=PLAIN are applied. For more information about authenticating with Confluent Cloud, see https://docs.confluent.io/platform/current/security/security_tutorial.html#overview. The sink scheme still supports non-authentication parameters such as topic_name and topic_prefix. It also supports the standard Kafka changefeed options (ex. kafka_sink_config). #111368

SQL language changes

• The RESTORE option strip_localities, which was added in #110606, has been renamed to remove_regions. This option will lead to a "region-less restore"; it is used to strip the locality and region information from a backup when there are mismatched cluster regions between the backup's cluster and the target cluster. Note that a restore using this option will fail if the backup's cluster had REGIONAL BY ROW table localities. This is because the RESTORE statement has a contract that all tables must be available to serve writes once it finishes. #111356
• Added initial support for executing the PLpgSQL OPEN statement, which allows a PLpgSQL routine to create a cursor. Currently, opening bound or unnamed cursors is not supported. In addition, OPEN statements cannot be used in a routine with an exception block. #110709
• Added support for declaring bound cursors, which associate a query with a cursor in a PLpgSQL routine before it is opened. #111092
• The SELECT FOR SHARE and SELECT FOR KEY SHARE statements previously did not acquire any locks. Users issuing these statements would expect them to acquire shared locks (multiple readers allowed, but no writers). This patch switches over the behavior to acquire such read locks when the user has selected the READ COMMITTED isolation level. For serializable transactions, we default to the previous behavior, unless the enable_shared_locking_for_serializable session setting is set to true. #109638
• When a PLpgSQL exception handler catches an error, it now rolls back any changes to database state that occurred within the block. Exception blocks are not currently permitted to catch 40001 and 40003 errors. #110998
• Added support for unnamed PLpgSQL cursors, which generate a unique name when no cursor name was specified. #111329
• Fixed a bug that caused CockroachDB to stop collecting new statistics about Statement fingerprints and Transaction fingerprints. #111613
• Make the max_event_frequency metric visible for public documentation and usage. This is the maximum event frequency at which we sample executions for telemetry. #111594

Operational changes

• Added the following metrics for Raft proposals and reproposals: raft.commands.proposed, raft.commands.reproposed.unchanged, and raft.commands.reproposed.new-lai. #111272
• Removed the cluster setting spanconfig.store.enabled and the ability to use the COCKROACH_DISABLE_SPAN_CONFIGS environment variable. #110253
• Renamed the metric fluent.sink.conn.errors to log.fluent.sink.conn.errors. The addition of the log. prefix was to better group together logging-related metrics. The behavior and purpose of the metric remains unchanged. #111126
• Set the Metric Type metadata on the metric log.fluent.sink.conn.errors. Previously, the Metric Type was incorrectly left unset. Note that this is an update to the metric's metadata; the behavior and purpose of the metric remains unchanged. #111126
• Added a new metric log.buffered.messages.dropped. Buffered network logging sinks have a max-buffer-size attribute, which determines, in bytes, how many log messages can be buffered. Any fluent-server or http-server log sink that makes use of a buffering attribute in its configuration (enabled by default) qualifies as a buffered network logging sink. If this buffer becomes full, and an additional log message is sent to the buffered log sink, the buffer would exceed this max-buffer-size. Therefore, the buffered log sink drops older messages in the buffer to handle, in order to make room for the new. log.buffered.messages.dropped counts the number of messages dropped from the buffer. Note that the count is shared across all buffered logging sinks. #111126
• Added the metric log.messages.count. This metric measures the count of messages logged on the node since startup. Note that this does not measure the fan-out of single log messages to the various configured logging sinks. This metric can be helpful in understanding log rates and volumes. #111126
• Added the file-based-headers field found in the http-defaults section of the log config, which accepts 'key-filepath' pairs. This allows values found at filepaths to be updated without restarting the cluster by sending SIGHUP to notify that values need to be refreshed. #111235
• Added the cluster setting kv.snapshot.ingest_as_write_threshold, which controls the size threshold below which snapshots are converted to regular writes. It defaults to 100KiB. #110943

Cluster virtualization

• The name of the virtual cluster that the SQL client is connected to can now be inspected via the SQL session variable virtual_cluster_name. #111565

Command-line changes

• The following cluster settings have been renamed; the previous names remain available for backward-compatibility. #109415

  Previous name	New Name
  server.shutdown.drain_wait	server.shutdown.initial_wait
  server.shutdown.lease_transfer_wait	server.shutdown.lease_transfer_iteration.timeout
  server.shutdown.query_wait	server.shutdown.transactions.timeout
  server.shutdown.connection_wait	server.shutdown.connections.timeout
  server.shutdown.jobs_wait	server.shutdown.jobs.timeout

DB Console changes

• Fixed an error on the SQL Activity page when there was a workload, and then the workload stopped so that no queries ran against the database in the last hour. #111420
• On the Metrics page, now the information about which metric is used to create each chart is available on the chart's tooltip. #111469

Bug fixes

• Fixed the error message that is returned when the user attempts to drop an ENUM value that is used at least twice in an ARRAY column. #111354
• Added a check for values before using mean on the Plan Details page, fixing a crash. #111472
• Fixed the metric name for Schema Registry Registrations on the Metrics page. #111469
• Fixed a panic that could occur if a query used a string larger than 2^31-1 bytes. This was triggered by attempting to import a 2.7 GiB CSV file. #111627
• Fixed a bug where atttypmod in pg_attribute was not populated for TIMESTAMP / INTERVAL types, which meant that ORMs could not know the precision of these types properly. #111400

Contributors

This release includes 130 merged PRs by 43 authors.

v23.2.0-alpha.2

Release Date: October 2, 2023

Downloads

Warning:

CockroachDB v23.2.0-alpha.2 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-alpha.2.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.2.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.2.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-alpha.2.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-alpha.2.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.2.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.2.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.2.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-alpha.2.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-alpha.2.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is Experimental and not yet qualified for production use and not eligible for support or uptime SLA commitments.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-alpha.2

Source tag

To view or download the source code for CockroachDB v23.2.0-alpha.2 on Github, visit v23.2.0-alpha.2 source tag.

Changelog

View a detailed changelog on GitHub: v23.2.0-alpha.1...v23.2.0-alpha.2

Security updates

• The SIGHUP signal now clears the cached expiration times for client certificates that are reported by the security.certificate.expiration.client metric. #110726

General changes

• Increased the maximum permitted value of the COCKROACH_RPC_INITIAL_WINDOW_SIZE environment variable to 64MB. In conjunction with tuning your operating system's maximum TCP window size, this can increase the throughput that Raft replication can sustain over high latency network links. #111255

SQL language changes

• The discard log message is now limited to once per minute by default. The message now includes both the number of transactions and the number of statements that were discarded. #110805
• The cluster setting kv.rangefeed.enabled no longer controls access to RANGEFEED SQL commands. Instead, use feature.changefeed.enabled. #110676
• SQL commands that were previously limited to the admin system privilege can now be used by users with the VIEWCLUSTERMETADATA or REPAIRCLUSTERMETADATA system privilege, depending on whether the operation is read-only or modifies state. #110084
• Added a last_error column to the cluster_execution_insights, node_execution_insights, cluster_txn_execution_insights, and node_txn_execution_insights tables. These columns contain error messages for failed executions. #110565
• The new backup option updates_cluster_monitoring_metrics tracks the timestamp of the last backup failure due to a KMS error. This option is disabled by default. #104634
• The new restore option strip_localities optionally strips the locality information from a backup when restoring to a cluster with different regions than the source cluster.

Restoring a cluster or database that contains regional-by-row tables, or restoring a regional-by-row table, requires you to modify the database:

• To restore a cluster with regional-by-row tables, you must drop the zone config of the database, then drop the type d.public.crdb_internal_region.
• To restore a database that contains regional-by-row tables, or to restore a regional-by-row table, you must drop the type d.public.crdb_internal_region.
• You must alter the crdb_region column to set the default region for newly-written rows.
• You must discard the previous zone config, which contains outdated information, such as that related to the partitions and constraints after the restore. This column specifies each row's home region and is a prefix to the table's primary key. Stripping localities does not modify this column, because it would require the entire table to be written.

This change is part of a larger effort, and this feature is subject to change.

#110606

• Added a check to disallow queries that use predicate locking, since explicit uniqueness checks are not yet supported under Read Committed isolation. INSERT, UPDATE, and UPSERT statements against some REGIONAL BY ROW tables will fail under Read Committed isolation with the following error:

  unimplemented: explicit unique checks are not yet supported under read committed isolation SQLSTATE: 0A000

For more details about which REGIONAL BY ROW tables are affected, refer to Issue #110873.

#110879 - The created field produced by SHOW STATISTICS has been updated from TIMESTAMP to TIMESTAMPTZ. Statistic creation times are now displayed in the session time zone if it is set. #110753

Operational changes

• Removed the node-level engine.stalls timeseries metric. This metric has not been updated for several releases. #110936

DB Console changes

• The legend is now always displayed on charts in DB Console Metrics pages. In addition, when you select an item from the legend that represents a single line in the chart, that line is selected in the chart. #110809
• When collecting a statement bundle, you can now filter by a specific plan gist or collect diagnostics for all plan gists. #110931
• Statement and Transaction detail pages now include an Error Message row. Users with the VIEWACTIVITY system privilege can view the full error message, and users with the VIEWACTIVTYREDACTED system privilege can view the redacted error message. If a user has both privileges, VIEWACTIVITYTREDACTED` takes precedence. #110849
• A new dashboard in the SQL Dashboard page tracks how often distributed queries with errors were rerun using the "rerun as local" mechanism, as well as how often those reruns failed. the number of times distributed queries that resulted in errors were rerun as local as well as when those reruns failed. The "rerun as local" mechanism is new in v23.2 and is enabled by default. For more information, contact your Cockroach Labs account representative. #110619
• The DB Console Insights page now shows the error message when a transaction fails at the COMMIT stage. #110898
• The Overload Dashboard page now includes the following graphs to monitor admission control:
  • IO Overload - Charts normalized metric based on admission control target thresholds. Replaces LSM L0 Health graph which used raw metrics.
  • KV Admission Slots Exhausted - Replaces KV Admission Slots graph.
  • Flow Tokens Wait Time: 75th percentile - Use to monitor the new replication admission control feature.
  • Requests Waiting For Flow Tokens - Use to monitor the new replication admission control feature.
  • Blocked Replication Streams - Use to monitor the new replication admission control feature. #110135

Bug fixes

• Fixed a race condition in the Replica lifecycle that could result in a failed SQL request when the request could have been successfully retried. #110806
• Fixed a bug where a CREATE TABLE command with an IDENTITY column did not properly propagate the type of the column into the sequence. #110621
• Fixed a panic when decoding a gist in a foreign database that does not contain a table referred to by the gist. #110966
• A synthetic dropped column have been added to the pg_attribute table. This column tracks the attribution numbers for dropped attributions, to work around issues with ORMs that are not designed to handle gaps in attribution numbering in the pg_attribute table. #111019
• Fixed a rare internal error in the unnest and information_schema._pg_expandarray built-in functions where passed string arguments could be cast to an array. #110956
• External connection URLs now accept the scheme azure-blob for connections to Azure Blob Storage and the scheme azure-kms for connections to Azure KMS. For backward compatibility, schemes azure and azure-storage schemes continue to work for connections to Azure Blob Storage. #111217
• Fixed a bug where vectorized COPY FROM could produce a plan with more than one RenderNodes, when only zero or one should be allowed. This could result in multiple render nodes in a table with a hash sharded primary key. #111284
• Fixed a bug in DB Console's Statement Diagnostic page that could cause the page to crash if the response was larger than 50 KB. The page now keeps pulling results until no maximum size errors are encountered. #111128
• Fixed a bug where DB Console instances proxied at different subpaths that use OIDC pointed to an incorrect relative OIDC login path. #111240
• Fixed a bug where changing the setting server.telemetry.hot_ranges_stats.interval had no effect. #111305

Performance improvements

• Fixed a performance bug that could result in rewriting a 128-MB file each time a store file is created, renamed, or removed when Encryption At Rest is enabled on a large store with many small files. #111069
• Improved compaction heuristics to mitigate read amplification growth and admission control throttling when processing large deletes, such as during node decommissioning, replica rebalancing, or when dropping tables. #111277

Contributors

This release includes 157 merged PRs by 54 authors.

v23.2.0-alpha.1

Release Date: September 26, 2023

Downloads

Warning:

CockroachDB v23.2.0-alpha.1 is a testing release. Testing releases are intended for testing and experimentation only, and are not qualified for production environments and not eligible for support or uptime SLA commitments.

Note:

Experimental downloads are not qualified for production use and not eligible for support or uptime SLA commitments, whether they are for testing releases or production releases.

Operating System	Architecture	Full executable	SQL-only executable
Linux	Intel	cockroach-v23.2.0-alpha.1.linux-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.1.linux-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.1.linux-arm64.tgz (Experimental) (SHA256)	cockroach-sql-v23.2.0-alpha.1.linux-arm64.tgz (Experimental) (SHA256)
Mac (Experimental)	Intel	cockroach-v23.2.0-alpha.1.darwin-10.9-amd64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.1.darwin-10.9-amd64.tgz (SHA256)
	ARM	cockroach-v23.2.0-alpha.1.darwin-11.0-arm64.tgz (SHA256)	cockroach-sql-v23.2.0-alpha.1.darwin-11.0-arm64.tgz (SHA256)
Windows (Experimental)	Intel	cockroach-v23.2.0-alpha.1.windows-6.2-amd64.zip (SHA256)	cockroach-sql-v23.2.0-alpha.1.windows-6.2-amd64.zip (SHA256)

Docker image

Multi-platform images include support for both Intel and ARM. Multi-platform images do not take up additional space on your Docker host.

Within the multi-platform image:

• The ARM image is Experimental and not yet qualified for production use and not eligible for support or uptime SLA commitments.
• The Intel image is Generally Available for production use.

To download the Docker image:

docker pull cockroachdb/cockroach-unstable:v23.2.0-alpha.1

Source tag

To view or download the source code for CockroachDB v23.2.0-alpha.1 on Github, visit v23.2.0-alpha.1 source tag.

Backward-incompatible changes

• The pre-v23.1 output produced by SHOW RANGES, crdb_internal.ranges, and crdb_internal.ranges_no_leases was deprecated in 23.1 and is now replaced by default with output that's compatible with coalesced ranges (i.e., ranges that pack multiple tables/indexes/partitions into individual ranges). See the v23.1 release notes for SHOW RANGES for more details. #102961
• When a deployment is configured to use a time zone (new feature) for log file output using formats crdb-v1 or crdb-v2, it becomes impossible to process the new output log files using the cockroach debug merge-logs command from a previous version. The newest cockroach debug merge-logs code must be used instead. #104265
• When customizing the SQL shell's interactive prompt, the special sequence %M now expands to the full host name instead of the combination of host name and port number. To include the port number explicitly, use %>. The special sequence %m now expands to the host name up to the first period. #105137
• The cockroach debug zip command stores data retrieved from SQL tables in the remote cluster using the TSV format by default. #107474
• The changefeed.protect_timestamp.max_age cluster setting will only apply to newly created changefeeds in v23.2. For existing changefeeds, you can set the protect_data_from_gc_on_pause option so that changefeeds do not experience infinite retries and accumulate protected change data. You can use the ALTER CHANGEFEED statement to add protect_data_from_gc_on_pause to existing changefeeds. #103539
• The changefeed.new_pubsub_sink_enabled cluster setting is now enabled by default, which improves changefeed throughput. With this setting enabled, the top-level fields in JSON-encoded messages are capitalized: {Key: ..., Value: ..., Topic: ...}. After upgrading to CockroachDB v23.2, you may need to reconfigure downstream systems to parse the new message format. If you disable this setting, changefeeds emitting to Pub/Sub sinks with JSON-encoded events have the top-level message fields all lowercase: {key: ..., value: ..., topic: ...}.

Security updates

• Users who have the CREATEROLE role option can now grant and revoke role membership in any non-admin role. This change also removes the sql.auth.createrole_allows_grant_role_membership.enabled cluster setting, which was added in v23.1. In v23.2, the cluster setting is effectively always true. #104376

General changes

• You can now set Docker command arguments using the COCKROACH_ARGS environment variable. #98899
• Extended the /api/v2/nodes API endpoint with a storeMetrics field. #98208
• CockroachDB would previously use separate ranges for each table, index, or partition. This is no longer true. It is possible now to have multiple tables, indexes, and partitions get packed into the same range. For users with many of these schema objects, this will reduce the total range count in their clusters. This is especially true if individual tables, indexes, or partitions are smaller than the default configured maximum range size (controlled using zone configs, specifically the range_max_bytes parameter). We made this change to improve scalability with respect to the number of schema objects, since the underlying range count is now no longer a bottleneck. Users upgrading from v22.2, when finalizing their upgrade, may observe a round of range merges and snapshot transfers (to power said range merges) as a result of this change. If users want to opt-out of this optimization, they can configure the following cluster setting: SET CLUSTER SETTING spanconfig.storage_coalesce_adjacent.enabled = false; #98820
• EXPORT INTO PARQUET will now use a new internal implementation for writing Parquet files using the Parquet spec version 2.6. There should be no significant impact to the structure of files being written. There is one minor change: all columns written to Parquet files will be nullable (i.e., the Parquet repetition type is OPTIONAL). #104234
• Spatial libraries for CockroachDB now rely on GEOS 3.11 instead of GEOS 3.8. #106642
• CockroachDB no longer distributes libgeos for the experimental Windows build. Users can instead install GEOS directly from the source: https://libgeos.org/usage/download/. #106642
• The Formatting of byte figures in Pebble logs has been improved. Tools that parse these logs might need updating. #107392
• CockroachDB now has a new CLI option, --experimental-shared-storage to rebalance data faster from node to node. #105839
• Fixed a bug where, internally, if we print a 0 decimal with a very low exponent we use excessive memory. This is not possible when using the DECIMAL type, but may be possible when using crdb_internal functions. #110527

Enterprise edition changes

• The kafka_sink_config Compression and RequiredAcks options are now case-insensitive. #100929
• Changefeeds emit significantly fewer duplicate messages during node and cluster restarts. #102717
• CockroachDB has a new changefeed.protect_timestamp.max_age setting (by default 4 days), which will cancel running changefeed jobs if they fail to make forward progress for a period of time. This setting is used if the explicit gc_protect_expires_after option is not set. In addition, the protect_data_from_gc_on_pause option has been deprecated. This option is no longer needed since changefeed jobs always protect data. #103539
• Changefeeds now officially support the Parquet format using specification version 2.6. It is only usable with the cloud storage sink. The syntax to use Parquet is: CREATE CHANGEFEED FOR foo INTO ... WITH format=parquet. It supports all standard changefeed options and features including CDC transformations, except it does not support the topic_in_value option. #104528
• Changefeeds that create files over an HTTP connection may now be specified using INTO 'file-https://' to disambiguate with webhook-https. #107572
• The pgcrypto functions encrypt, encrypt_iv, decrypt, and decrypt_iv are now implemented. These functions require an enterprise license on a CCL distribution. #105654
• CockroachDB now paces the rangefeed goroutine creation rate to improve scheduler latency. This improves observability by adding an additional column in the crdb_internal.active_rangefeed table to indicate if the range is currently in catchup scan mode. #109346

SQL language changes

• Fixed the helper message on UPDATE SQL statements to include the optional FROM cause. #98709
• CockroachDB now supports enabling forward indexes and ordering on JSON values. #99275

• Added a new column visibility to crdb_internal.table_indexes and information_schema.statistics. Also added a new column visibility to the output of following SQL statements:

  SHOW INDEX FROM (table_name);
  

  SHOW INDEXES FROM (table_name);
  

  SHOW KEYS FROM (table_name);
  

  SHOW INDEX FROM DATABASE (database_name);
  

  SHOW INDEXES FROM DATABASE (database_name);
  

  SHOW KEYS FROM DATABASE (database_name);
  

  This new column contains a floating point number specifying the level of visibility of the index, from 0 (not visible) to 1 (fully visible). If the value is between 0 and 1, the index will be visible to the corresponding fraction of queries. #101334

• ALTER INDEX ... VISIBILITY ... is now supported. It can change an index visibility to any visibility between 0.0 and 1.0. Visibility 0.0 means the index is not visible to the optimizer, while visibility 1.0 means the index is fully visible. A value in the range between 0.0 and 1.0 means the index will be visible to the corresponding fraction of queries. #87301

• CockroachDB now has support for non-aggregate expressions involving columns outside of the grouping columns when the grouping columns include all key columns of a unique index and those key columns are not nullable. #101675

• CockroachDB now supports CREATE INDEX ... VISIBILITY ... and CREATE TABLE ... (... INDEX (...) VISIBILITY ...). This allows users to set the index visibility to any visibility between 0.0 and 1.0. Visibility 0.0 means the index is not visible to the optimizer, while visibility 1.0 means the index is fully visible. A value in the range between 0.0 and 1.0 means the index will be visible to the corresponding fraction of queries. #101812

• Row level TTL now supports DESC order primary key columns. #101869

• Added the ST_BdPolyFromText built-in which copies the behavior of the PostGIS function. Takes in only a multilinestring geometry and returns a polygon. It will return an error if anything other than a multilinestring is input, and will return an error if internally a multipolygon is created for some reason. NULL inputs also return NULL. #102708

• SHOW SCHEDULES now shows the schedule options with which the schedules were created. SHOW SCHEDULES FOR BACKUP additionally shows if the schedule is a full or incremental backup schedule. #102890

• You can no longer use PREPARE with EXPLAIN ANALYZE statements. Previously, this was allowed, but attempts to EXECUTE the prepared EXPLAIN ANALYZE statements would result in an error. #103259

• ttl_expiration_expression now allows stable operators and functions. This allows intervals to be directly added to TIMESTAMPTZ expressions. See https://www.postgresql.org/docs/15/xfunc-volatility.html. #102974

• CockroachDB now allows INSERT commands in UDF statement bodies. #102773

• CockroachDB now allows UPDATE and UPSERT commands in UDF statement bodies. #102773

• The READ COMMITTED isolation level is now supported. It can be used in the following ways:

  • When starting a transaction, use BEGIN ISOLATION LEVEL READ COMMITTED.
  • After starting a transaction, but before performing reads or writes, use SET TRANSACTION ISOLATION READ COMMITTED.
  • Configure it as the default isolation level using the default_transaction_isolation session variable. To see the isolation level of the currently running transaction, use either SHOW TRANSACTION ISOLATION LEVEL or SHOW transaction_isolation.

  #103482

• Added version gates which require all nodes in a given cluster to have a minimum binary version number, which in turn is required for creating forward indexes on JSON columns and for ordering JSON columns. #101932

• CockroachDB now allows DELETE commands in UDF statement bodies. #103531

• Added a new cluster setting sql.auth.public_schema_create_privilege.enabled which controls whether users receive CREATE privileges on the public schema or not. The setting applies at the time that the public schema is created, which happens whenever a database is created. The setting is true by default. #103598

• EXPLAIN (DDL) statements now have descriptor, index, column, constraint, and other ID values decorated with names when available. There is now also a new EXPLAIN (DDL, SHAPE) statement that provides information on costly operations planned by the declarative schema changer, like which index backfills and validations will get performed. #103930

• A new statistic KV pairs read is now exposed on EXPLAIN ANALYZE output in some cases (when this number is different from the KV rows read statistic or when the VERBOSE option is requested). This new statistic is also added to the telemetry sampled query events. #104079

• The KV rows read statistic in EXPLAIN ANALYZE output has been renamed to KV rows decoded to better reflect its meaning. #104079

• Table names are now allowed in SELECT lists inside view and UDF definitions. #104929

• SHOW JOB WITH EXECUTION DETAILS for a backup job will regenerate the DistSQL plan diagram with per-node and per-processor progress information. This will help users better understand the state of a running backup job. #103145

• The crdb_internal.node_transactions and crdb_internal.cluster_transactions tables now have columns for isolation_level, priority, and quality_of_service. #105009

• The SHOW RANGES command will now emit span statistics when the DETAILS option is specified. The statistics are included in a new column named span_stats, as a JSON object. The statistics are calculated for the identifier of each row. SHOW RANGES WITH DETAILS will compute span statistics for each range. SHOW RANGES WITH TABLES, DETAILS will compute span statistics for each table, and so on. The span_stats JSON object has the following keys:

  • approximate_disk_bytes
  • [key|val|sys|live|intent]_count
  • [key|val|sys|live|intent]_bytes

  approximate_disk_bytes is an approximation of the total on-disk size of the given object.

  key_count is the number of meta keys tracked under key_bytes. key_bytes is the number of bytes stored in all non-system point keys, including live, meta, old, and deleted keys. Only meta keys really account for the "full" key; value keys only for the timestamp suffix.

  val_count is the number of meta values tracked under val_bytes. val_bytes is the number of bytes in all non-system version values, including meta values.

  sys_count is the number of meta keys tracked under sys_bytes. sys_bytes is the number of bytes stored in system-local key-value pairs. This tracks the same quantity as (key_bytes + val_bytes), but for system-local metadata keys (which aren't counted in either key_bytes or val_bytes).

  live_count is the number of meta keys tracked under live_bytes. live_bytes is the number of bytes stored in keys and values which can in principle be read by means of a Scan or Get in the far future, including intents but not deletion tombstones (or their intents). Note that the size of the meta key-value pair (which could be explicit or implicit) is included in this. Only the meta key-value pair counts for the actual length of the encoded key (regular pairs only count the timestamp suffix).

  intent_count is the number of keys tracked under intent_bytes. It is equal to the number of meta keys in the system with a non-empty Transaction proto. intent_bytes is the number of bytes in intent key-value pairs (without their meta keys).

  #103128

• Introduced the pg_lsn data type, which is used to store the lsn associated with replication. #105031

• Users now can issue one ALTER TABLE statement with a combination of any number of ADD COLUMN, any number of DROP COLUMN, one ALTER PRIMARY KEY, and any number of ADD CONSTRAINT clauses. For example, with this PR, we now support statements like:

  CREATE TABLE t (i INT PRIMARY KEY, j INT NOT NULL, k INT NOT NULL);  ALTER TABLE t ADD COLUMN p INT DEFAULT 30, ALTER PRIMARY KEY USING COLUMNS (j), DROP COLUMN k, ADD CHECK (i > 0);
  

  #99526

• Added the ability to add numeric values to LSNs, or sub a decimal value from a LSN. #105326

• Implemented the pg_lsn - pg_lsn = decimal built-in function, which subtracts 2 LSNs to return a decimal. #105326

• Added limited support for scalar PL/pgSQL functions. Supported statements are variable declarations, variable assignments, IF statements, simple LOOP statements (with no conditions), EXIT and CONTINUE statements, and RETURN statements. #104755

• Implemented the spatial built-in ST_AsMVTGeom. #105530

• Pg_class's relreplident field was previously unpopulated. It is now populated with d for all tables (as each table has a primary key) and n otherwise. #106242

• Added the pg_sequence_last_value built-in function, which returns the last value generated by the sequence. #106445

• RESTORE can now be passed a WITH EXECUTION LOCALITY option similar to BACKUP, to restrict execution of the job to nodes with matching localities. #104439

• Added the REPLICATION user role option, which allows a user to use the streaming replication protocol. There is a corresponding REPLICATION system privilege. #106082

• A new view-only session variable, max_connections was added. This can be used with SHOW to view the maximum amount of non-superuser SQL connections allowed at a given time. #106952

• Added the nameconcatoid built-in function, which concatenates a name with an OID. #105944

• The pg_catalog.pg_language table is now populated with data about the languages used to define functions. #105944

• The information_schema.routines view is now populated with information about functions. #105944

• The information_schema.parameters table is now populated with information about function parameters. #105944

• Added support for the PLpgSQL RAISE statement, which allows sending notices to the client and raising errors. Currently the notice is only sent to the client. Support for logging notices will be added in a future release. #106351

• The public pseudo-role now receives the EXECUTE privilege by default for all user-defined functions that are created. This can be adjusted by using ALTER DEFAULT PRIVILEGES. #107317

• The crdb_interanal.node_statement_statistics table redacts the error message if the user has the VIEWACTIVITYREDACTED privilege, and does not redact the error message if the user has VIEWACTIVITY. If the user has both, VIEWACTIVITYREDACTED takes precedence and the last error is redacted. #107076

• The crdb_internal.cluster_locks table now has a isolation_level column indicating the isolation level. #107309

• In CommonSQLExecDetails, which is emitted as part of the SQL audit logs, SQL exec logs, and telemetry events, there is a new field: StmtPosInTxn. It represents the statement's index in the transaction, starting at 1. #107081

• cluster_logical_timestamp now returns an error when called at isolation levels lower than SERIALIZABLE. #107090

• EXPLAIN ANALYZE output now includes:

  • The isolation level of the statement's transaction.
  • The priority of the statement's transaction.
  • The quality of service level of the statement's transaction.

  #105857

• Added a new session variable, enable_implicit_fk_locking_for_serializable, which controls locking during foreign key checks under SERIALIZABLE isolation. With this set to true, foreign key checks of the referenced (parent) table, such as those performed during an INSERT or UPDATE of the referencing (child) table, will lock the referenced row using SELECT FOR SHARE locking. This is somewhat analogous to the existing enable_implicit_select_for_update variable but applies to the foreign key checks of a mutation statement instead of the initial row fetch. Under weaker isolation levels such as read committed, SELECT FOR SHARE locking will always be used to ensure the database maintains the foreign key constraint, regardless of the current setting of enable_implicit_fk_locking_for_serializable. #105857

• Add a new session variable, enable_durable_locking_for_serializable, which controls locking durability under SERIALIZABLE isolation. With this set to true, SELECT FOR UPDATE locks, SELECT FOR SHARED locks, and constraint check locks (e.g., locks acquired during foreign key checks if enable_implicit_fk_locking_for_serializable is set to true) will be guaranteed-durable under serializable isolation, meaning they will always be held to transaction commit. These locks are always guaranteed-durable under weaker isolation levels. By default, under serializable isolation these locks are best-effort rather than guaranteed-durable, meaning in some cases (e.g., leaseholder transfer, node loss, etc.) they could be released before the transaction commits. Serializable isolation does not rely on locking for correctness, only using it to improve performance under contention, so this default is a deliberate choice to avoid the performance overhead of lock replication. #107749

• The cluster setting server.cpu_profile.enabled has been removed. server.cpu_profile.cpu_usage_combined_threshold can enable and disable CPU profiling. #107717

• Added support for CONSTANT variable declarations in PLpgSQL routines. Any assignment to a variable declared with the CONSTANT keyword will raise a compile-time error. #107682

• Added a new syntax to SHOW DEFAULT PRIVILEGES, SHOW DEFAULT PRIVILEGES FOR GRANTEE <grantee>, that shows the default privileges that a grantee received. #107953

• The Statement diagnostics feature has been extended to support collecting a bundle for a particular plan. Namely, the existing fingerprint-based matching has been extended to also include plan-gist-based matching. Such bundles will miss a couple of things: plan.txt file as well as the tracing of the optimizer. At the moment, the feature is only exposed via an overload to the crdb_internal.request_statement_bundle built-in function. We now also support "anti-match": collecting a bundle for any plan other than the provided plan gist. #105477

• SHOW BACKUP's timestamp columns are now TIMESTAMPTZ, meaning they render in the session offset. #108290

• Attempting to drop a column when safe updates are enabled (sql_safe_updates = on) now additionally warns users that indexes referencing that column will be automatically dropped. #108047

• NOTICEs are now emitted for each index dropped by an ALTER TABLE ... DROP COLUMN ... statement. #108047

• SHOW JOBS now returns times (created, last_run, and so on) using the TIMESTAMPTZ column type instead of the TIMESTAMP type, meaning they are now rendered using the session offset. #108353

• Added a cluster setting sql.schema.force_declarative_statements to enable/disable DDL in the declarative schema changer. #107815

• Added the new built-in functions workload_index_recs() and workload_index_recs(TIMESTAMPTZ), which return workload level index recommendations (columns of string, each string represent an index recommendation) from statement level index recommendations (as candidates) in system.statement_statistics. If the TIMESTAMPTZ is given, it will only consider those candidates generated after that TIMESTAMPTZ value. #106525

• Added support for specifying PLpgSQL IF statements with ELSIF branches. #108211

• The admin API database details endpoint now returns authoritative range statistics. #108037

• Added the max_retries_for_read_committed session variable. It defaults to 10, and determines the number of times an individual statement in an explicit READ COMMITTED transaction will be retried if it encounters a retryable transaction error. #107044

• Added support for the execution of PLpgSQL functions with exception blocks. This allows a PLpgSQL function to catch and handle arbitrary errors it encounters during its execution. #107601

• Added the built-in functions bitmask_or, bitmask_and and bitmask_xor for variable-length input bitwise OR, AND, and XOR operations, respectively. #107863

• The oidvectortypes built-in has been implemented, which can format oidvector. #108467

• Added support for executing SQL statements directly within PLpgSQL routines. Note that this currently only applies to the subset of statements that can be executed within SQL UDFs, so CREATE TABLE is not supported, for example. INTO syntax is also supported. For example, SELECT * INTO a, b FROM xy;. #107920

• A SQL client can now request strict atomicity for mixed DDL/DML transactions with the new session variable strict_ddl_atomicity, which defaults to false. When this variable is set to true, CockroachDB will refuse to accept processing those specific DDL statements inside BEGIN...COMMIT for which it cannot guarantee atomic processing (other DDL statements are still allowed). Note that schema changes implicit in certain operations (e.g., IMPORT) are not protected via the new mechanism and can still fail with XXA00 errors. #42063

• Fixed an issue where the UI was missing query text and details on the SQL Activity Transactions page if there were more than 500 transactions or statements. The statement_activity table now includes all statements for a transaction that are in the transaction_activity table. #109424

• Added the VIEWSYSTEMTABLE system privilege. Users with this privilege have SELECT privileges for all tables in the system database. #109474

• The statement_activity and transaction_activity tables column execution_total_cluster_seconds is now accurate. The combinedstmts endpoint returns the correct value for the StmtsTotalRuntimeSecs and TxnsTotalRuntimeSecs properties. #109592

• The persistedsqlstats table maximum size check is now done once an hour instead of every 10 minutes. This reduces the risk of serialization errors on the statistics tables. #109696

• The deprecated session variable idle_in_session_timeout is now hidden from introspection. It was previously changed to idle_session_timeout. #109872

• The session variable ssl is now visible through introspection for better compatibility with PostgreSQL. #109872

• The session variable session_user is now invisible through introspection, in a way consistent with session_authorization and PostgreSQL. #109872

• There is now a CREATEROLE system privilege, which is analogous to the existing CREATEROLE role option, but can also be inherited by role membership. #109258

• Added the gen_random_bytes built-in function, which generates cryptographically secure random bytes. #110107

• The hash function used by hash-sharded indexes was changed to mod(fnv32(md5(crdb_internal.datums_to_bytes(columns))), bucket_count). Previously, it did not use md5. This change was made to enhance the uniformity of bucket distribution in cases when the bucket count is a power of 2, and the columns being sharded have numerical properties that make the fnv32 function return values with a non-uniformly distributed modulus. #109374

• New datetime built-ins (make_date, make_timestamp, and make_timestamptz) have been added, allowing for the creation of timestamps, timestamps with time zones, and dates. In addition, date_trunc now allows for a timestamp to be truncated in a specified timezone (to a specified precision). #108824

• There is now a CREATELOGIN system privilege, which is analogous to the existing CREATELOGIN role option, but can also be inherited by role membership. #110220

• There is now a CREATEDB system privilege, which is analogous to the existing CREATEDB role option, but can also be inherited by role membership. #110220

• There is now a CONTROLJOB system privilege, which is analogous to the existing CONTROLJOB role option, but can also be inherited by role membership. #110220

• The persistedsqlstats table maximum size check is now done once an hour instead of every 10 minutes. This reduces the risk of serialization errors on the statistics tables. #110173

• The new cluster setting sql.txn.read_committed_syntax.enabled, controls whether transactions run under READ COMMITTED or SERIALIZABLE isolation. It defaults to false. When set to true, the following statements will configure transactions to run under READ COMMITTED isolation:

  • BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED
  • SET TRANSACTION ISOLATION LEVEL READ COMMITTED
  • SET default_transaction_isolation = 'read committed'
  • SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL READ COMMITTED

  #110624

• The cluster setting sql.metrics.statement_details.gateway_node.enabled now defaults to false, to reduce the number of rows generated in SQL Statistics pages. #107788

• The default value for the ttl_job_cron table storage parameter is now @daily rather than @hourly. This parameter controls the default recurrence of the row-level TTL job. As part of this change, the output of the SHOW CREATE TABLE statements now include the ttl_cron_job parameter only if it is explicitly set. #110623

Operational changes

• Removed a timeseries metric that has not been reported for several versions. #100524
• Added two new metrics, range.snapshots.(send|recv)-queue-bytes, to track the total size of all snapshots waiting in the snapshot queue. #100942
• Exposed a new metric storage.compactions.duration, computed by the storage engine, that provides the cumulative time the storage engine has spent in compactions. This duration may exceed time elapsed, because of concurrent compactions, and may be useful in monitoring compaction concurrency. #103670
• Two new store metrics, range.snapshots.cross-region.sent-bytes and range.snapshots.cross-region.rcvd-bytes, were added to track the aggregate of snapshot bytes sent from and received at a store across different regions. Note that these metrics require the nodes' localities to include a “region” tier key. If a node lacks this key but is involved in cross-region batch activities, an error message will be logged. #104111

• Added new store metrics to track the aggregate of snapshot bytes sent from and received at a store across different zones.

  • range.snapshots.cross-zone.sent-bytes
  • range.snapshots.cross-zone.rcvd-bytes

  For accurate metrics, follow these recommendations: - Configure region and zone tier keys consistently across nodes. - Within a node locality, ensure unique region and zone tier keys. - Maintain consistent configuration of region and zone tiers across nodes.

  #104417

• Added new store metrics:

  • raft.rcvd.bytes
  • raft.sent.bytes
  • raft.rcvd.cross_region.bytes
  • raft.sent.cross_region.bytes
  • raft.rcvd.cross_zone.bytes
  • raft.sent.cross_zone.bytes

  #105122

• Added new DistSender metrics:

  • distsender.batch_requests.replica_addressed.bytes
  • distsender.batch_responses.replica_addressed.bytes
  • distsender.batch_requests.cross_region.bytes
  • distsender.batch_responses.cross_region.bytes
  • distsender.batch_requests.cross_zone.bytes
  • distsender.batch_responses.cross_zone.bytes.

  #103963

• Added new Node metrics:

  • batch_requests.bytes
  • batch_responses.bytes
  • batch_requests.cross_region.bytes
  • batch_responses.cross_region.bytes
  • batch_requests.cross_zone.bytes
  • batch_responses.cross_zone.bytes

  #104585

• Added new RPC metrics to help you to diagnose RPC connection issues:

  • grpc.connection.avg_round_trip_latency
  • rpc.connection.failures
  • rpc.connection.healthy
  • rpc.connection.healthy_nanos
  • rpc.connection.heartbeats
  • rpc.connection.unhealthy
  • rpc.connection.unhealthy_nanos

  #99191

• Added a new metric changefeed.lagging_ranges that shows the number of ranges which are behind in changefeeds. This metric can be used with the metrics_label changefeed option. Added a new changefeed option lagging_ranges_threshold, which is the amount of time a range needs to be behind to be considered lagging. By default this is 3 minutes. Added a new option lagging_ranges_polling_interval, which controls how often the lagging ranges calculation is done. This setting defaults to polling every 1 minute. Note that polling adds latency to the metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update for an additional minute afterwards. Also note that ranges undergoing an initial scan for longer than the threshold are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. However, as ranges complete the initial scan, the number of ranges will decrease. #109835

• A histogram metric raft.replication.latency was added. It tracks the time between evaluation and application of the command. This includes time spent in the quota pool, in replication (including re-proposals) as well as log application, but notably not sequencing latency (i.e., contention and latch acquisition). #106094

• The default Raft scheduler concurrency cap has been increased from 96 to 128 workers, scaling with 8 workers per CPU up to the cap. The scheduler concurrency can be controlled using the COCKROACH_SCHEDULER_CONCURRENCY environment variable. #105521

• The new cluster setting server.hot_ranges_request.node.timeout controls the maximum amount of time that a hot ranges request will spend waiting for a node to provide a response. It defaults to 5 minutes. To disable timeouts, set it to 0. #107796

• Two new cluster settings control whether intent resolution is subject to admission control: kv.intent_resolver.send_immediately.bypass_admission_control.enabled and kv.intent_resolver.batch.bypass_admission_control.enabled. #109932

• The new cluster setting admission.l0_min_size_per_sub_level reduces the probability of admission control throttling when there is a sequence of small memtable flushes or small files ingested into L0. #109332

• The new cluster setting kv.intent_resolver.batcher.in_flight_backpressure_limit.enabled controls whether an in-flight RPC limit is enforced on intent resolution RPCs. It defaults to false. #109899

• BACKUP now skips contacting the ranges for tables on which exclude_data_from_backup is set, and can thus succeed even if an excluded table is unavailable. #108627

• Span stats requests will return a partial result if the request encounters any errors. Errors that would have previously terminated the request are now included in the response. #108456

• The rangefeed closed timestamp interval controlled by kv.rangefeed.closed_timestamp_refresh_interval now defaults to 3 seconds. This affects how often rangefeeds emit resolved timestamps, and thus how often changefeeds can emit checkpoints. Previously, its default value of 0 would fall back to kv.closed_timestamp.side_transport_interval, which defaults to 200 milliseconds. Users who rely on the setting kv.closed_timestamp.side_transport_interval to control the rangefeed closed timestamp interval should make sure they either set kv.rangefeed.closed_timestamp_refresh_interval to 0 to retain the old behavior (preferably before upgrading), or to an appropriate value. #108667

• The default value of timeout for http-servers logging sinks has been changed from 0 (i.e., "no timeout") to 2s. This is reflected in the http-defaults section of the log configuration. Users still maintain the ability to override the timeout, or disable it by explicitly setting it to 0 (e.g. timeout: 0). #109264

• Changefeed metrics now include a changefeed.checkpoint_progress metric which is similar to changefeed.max_behind_nanos but supports metrics labels, as well as a changefeed.aggregator_progress metric which can track the progress of individual aggregators (the lowest timestamp for which all aggregators with the label have emitted all values they're responsible for). #108757

• Added support for Prometheus native histograms behind an environment variable flag. #104302

• Requests for database details or table details from the UI, or usages of [SHOW RANGES WITH DETAILS]/docs/v23.2/show-ranges.html are no longer subject to errors if the number of requested spans is too large. #109464

• The cockroach debug zip command now has an option to omit goroutine stack dumps. This impacts the creation of nodes/*/stacks.txt and nodes/*/stacks_with_labels.txt within debug ZIP bundles. Users can opt to exclude these goroutine stacks by using the --include-goroutine-stacks=false flag. Note that fetching stack traces for all goroutines is a "stop-the-world" operation, which can momentarily have negative impacts on SQL service latency. Note also that any periodic goroutine dumps previously taken on the node will still be included in nodes/*/goroutines/*.txt.gz, as these would have already been generated and don't require any stop-the-world operations. #110177

• New rangefeed metrics help to troubleshoot rangefeed restarts. The metric names have the format distsender.rangefeed.retry.{reason}. #109346

• Rangefeeds regularly attempt to push long-running transactions to a future timestamp in order to emit checkpoints. The interval at which this is attempted has been increased from 250 milliseconds to 1 seconds. This is now configurable via the environment variable COCKROACH_RANGEFEED_PUSH_TXNS_INTERVAL. #110332

Cluster virtualization

When cluster virtualization is enabled:

• A selection box displays in DB Console Metrics pages when you are connected to the system virtual cluster, and allows you to view metrics for a specific virtual cluster. #103308
• A "no data" empty graph state has been added when switching to a virtual cluster with no data. #103971
• A selection box displays on custom charts in the DB Console and allows you to select a specific virtual cluster. #103780
• The name of the virtual cluster, when known, is now reported in logging events. #108807
• When cockroach debug zip is run for a cluster with virtualization enabled, data about virtual clusters is now stored in a virtual subdirectory rather than a tenants subdirectory. #106117
• When cluster virtualization is enabled, the following closed timestamp side-transport settings can be set only from the system virtual cluster: kv.closed_timestamp.target_duration, kv.closed_timestamp.side_transport_interval, and kv.closed_timestamp.lead_for_global_reads_override. #108678

Command-line changes

• The CLI commands that output SQL data now support the JSON output format (--format=json), in addition to newline-delimited JSON (ND-JSON, --format=ndjson) that had been supported since v22.2. #102595
• cockroach debug zip now supports the command-line flag --format to select the format used to store SQL table data, in the same way as cockroach sql. In contrast to cockroach sql however, its default value is json (resulting in files named .json) and the default is not dependent on whether the terminal is interactive. #102607

• The SQL shell now supports argument quoting for client-side commands in a similar way to psql: inside single quotes, \ can escape characters and recognize octal/hexadecimal sequences; and inside double quotes characters are passed through. The quote characters themselves, when doubled, result in themselves as part of the string.

  For example, the following commands both result in a SQL prompt that says go "world":

  \set prompt1 'go "world"'
  \set prompt1 go' '"world"
  

  To add color to the prompt:

  \set prompt1 '\033[34mmydb>\033[m'
  

  These quoting rules are similar to PostgreSQL, but are different from the rules used by POSIX shells and of other programming languages like Python or Go. For example, octal and hex escape sequences support a variable number of digits, and double quoted strings preserve the surrounding quotes. When in doubt, refer to the PostgreSQL documentation.

  #104610

• The configuration for log output sinks now accepts a new format-options field. This can be used to customize the output of a given format. Each format accepts different options. One available option for the json output format is datetime-format.

  For example:

  sinks:
    fluent-groups:
      custom-json:
        format: json
        format-options: {datetime-format: rfc3339}
  

  This introduces a (new) field datetime in each output JSON event, with the format specified by the option. As of this writing, the following values are documented:

  • none: disable the creation of the datetime field. This is the default value.
  • iso8601 / rfc3339: format the time stamp like "2006-01-02T15:04:05.999999999Z".
  • rfc1123: format the time stamp like "Mon, 02 Jan 2006 15:04:05 +0000".

  Enabling the datetime field introduces CPU overhead and is not recommended. When using output to a log collector such as Fluent or Datadog, the log collector can be configured to transform the timestamp provided by CockroachDB without requiring participation from CockroachDB itself. When inspecting a log file containing JSON output produced by CockroachDB, the command cockroach debug merge-log can consume the JSON data and reformat it using the crdb-v2 format which also includes the date and time using the RFC3339 format. #104265

• The json log output format now recognizes the extra format option datetime-timezone which selects which timezone to use when formatting the datetime field. datetime-timezone must be combined with datetime-format because the default value for the latter option is none (i.e., datetime is not produced by default). For example:

  sinks:
    fluent-groups:
      custom-json:
        format: json
        format-options: {datetime-format: rfc3339, datetime-timezone: America/New_York}
  

  #104265

• The json log format now recognizes the format options tag-style and fluent-tag. The existing formats json-compact, json-fluent, json-fluent-compact have been redefined to become aliases for json with different defaults for the two new options. #104265

• The crdb-v1 log format now recognizes the format options show-counter and colors. The existing formats crdb-v1-tty, crdb-v1-count, crdb-v1-tty-count have been redefined to become aliases for crdb-v1 with different defaults for the two new options. #104265

• The crdb-v2 log format now recognizes the format option colors. The existing formats crdb-v2-tty has been redefined to become aliases for crdb-v2 with a different default for the new option. #104265

• The log output formats crdb-v1 and crdb-v2 now support the format option timezone. When specified, the corresponding time zone is used to produce the timestamp column. For example:

  file-defaults:
      format: crdb-v2
      format-options: {timezone: america/new_york}
  

  Example logging output:

  I230606 12:43:01.553407-040000 1 1@cli/start.go:575 ⋮ [n?] 4  soft memory limit of Go runtime is set to 35 GiB                        ^^^^^^^ indicates GMT-4 was used
  

  The timezone offset is also always included in the format if it is not zero (e.g., for non-UTC time zones). This is necessary to ensure that the times can be read back precisely. #104265

• The command cockroach debug merge-log was adapted to understand time zones in input files read with format crdb-v1 or crdb-v2. #104265

• When customizing the SQL interactive prompt, %M and %m now behave more like psql when connecting over a Unix datagram socket. #105137

• The default value of the --format parameter to cockroach debug zip is tsv, like other CLI commands that can extract SQL data. #107474

• The debug.zip now includes the crdb_internal.probe_range table with a limit of 100 rows to prevent the query from taking too long. #107720

• The default value for the --max-sql-memory parameter of the cockroach demo command has been increased from 128 MiB to 256 MiB. #103642

• The command \demo recommission has been removed from cockroach demo. It had been obsolete and non-functional ever since v20.2. #108566

• Added limited statement_statistics to the debug ZIP file. #108210

• The following user-visible cluster settings have been renamed. The previous name is still available for backward compatibility.

  Previous name	New name
  server.web_session_timeout	server.web_session.timeout
  kv.closed_timestamp.follower_reads_enabled	kv.closed_timestamp.follower_reads.enabled
  kv.range_split.by_load_enabled	kv.range_split.by_load.enabled
  changefeed.balance_range_distribution.enable	changefeed.balance_range_distribution.enabled
  changefeed.batch_reduction_retry_enabled	changefeed.batch_reduction_retry.enabled
  server.clock.forward_jump_check_enabled	server.clock.forward_jump_check.enabled
  server.oidc_authentication.autologin	server.oidc_authentication.autologin.enabled
  sql.metrics.statement_details.dump_to_logs	sql.metrics.statement_details.dump_to_logs.enabled
  sql.trace.log_statement_execute	sql.log.all_statements.enabled
  trace.debug.enable	trace.http_debug_endpoint.enabled

  #109074

• The following cluster settings have been renamed. The previous names are available for backward-compatibility.

  Previous name	New name
  spanconfig.tenant_coalesce_adjacent.enabled	spanconfig.range_coalescing.application.enabled
  spanconfig.storage_coalesce_adjacent.enabled	spanconfig.range_coalescing.system.enabled

  #109077

• The new cockroach gen metric-list command generates metadata that describes the various metrics collected by an idle server. The list does not include dynamic metric names whose names are generated based on the workload. #109042

DB Console changes

• The time window selection for metrics charts is now encoded in the URL via query params. #101258
• The Job Details page now has a tabbed UI that will allow users to toggle between the Overview and other future views for advanced debugging and observability. #102737
• Renamed "recent executions" to "active executions" in the UI. #103784
• The Changefeed Dashboard has been updated with new graphs to track backfill progress, protected timestamps age, and the number of schema registry registrations. The updates include renaming the Sink Byte Traffic graph to Emitted Bytes and the Max Changefeed Latency graph to Max Checkpoint Latency. #101790

• A new Networking tab has been added to the DB Console metrics dashboard. Metrics for network bytes sent and received are now displayed in the Networking tab rather than the Hardware tab. In addition, the following metrics have been added:

  • cr.node.round-trip-latency-p50
  • cr.node.round-trip-latency-p99
  • cr.node.rpc.connection.unhealthy

#104394

• The Job Details page now has a profiler tab for more advanced observability into a job's execution. Currently, we support collecting a cluster-wide CPU profile of the job. #103945
• The active executions views in the SQL Activity pages now support toggling between automatic and manual refresh. A manual refresh button was also added along with a timestamp indicating when the last refresh was performed. #103786
• The visibility of the cluster setting ui.display_timezone has been set to public. Documentation of the cluster setting has been added. No functionality has been changed. #106530
• Added a table in the Profiler job details page that lists all the available files describing a job's execution details #106879
• Add columns for p50, p90, p99 percentiles and latency min and max on Explain Plan tab on the Statement Execution Details page. #107719
• Fixed a broken query for the database details page that was causing an infinite loading state. #107893
• Added summary cards with total/average values for statistics on the Statement Execution Details page. #109056
• The DB Console now Shows a warning when the time period selected on SQL Activity pages is older than the oldest data available. #109164
• Users without the VIEWCLUSTERSETTINGS permission but with VIEWACTIVITY or VIEWACTIVITYREDACTED can now see index recommendations. #109047
• The DB Console now allows non-admin users to view the Databases page. #109245
• Non-admin users are able to use the Database Details page. #109432
• Non-admin users are able to use the Database Table page. #109521
• The "SQL Connection Rate" metric on the SQL Dashboard is downsampled using the MAX function instead of SUM. This improves situations where zooming out would cause the connection rate to increase for downsampled data. #110391

Bug fixes

• Fixed an internal error that can occur when CREATE OR REPLACE VIEW replaces a view with fewer columns and another entity depended on the view. #99057
• If views are created with circular dependencies, CockroachDB now returns an error (cyclic view dependency for relation) instead of crashing the node. This bug was present since at least 21.1. #99174
• Fixed a potential bug whereby a failed or cancelled IMPORT could in some cases leave some of the imported rows behind after it was cancelled, in the rare event that the writing processes were slow enough to continue writing after the cleanup process started. #97071
• Fixed a very rare bug that could cause keys to get unexpectedly deleted when rebalances occurred in a write-heavy workload. #102164
• It is now possible to properly redirect the output of SQL queries using the ndjson output table format in cockroach sql. This bug had been introduced in v22.2. #102595
• The unaccent built-in function no longer removes spaces. #103819
• The details of errors pertaining to invalid descriptors are not included any more in redacted debug ZIP files. #104050
• Fixed a bug where join expressions were processed incorrectly. #103782
• Fixed a bug that could cause a UDF to return a value that does not conform to the return type of the UDF. This bug was only present for UDFs that return user-defined types. The bug was present since v23.1. #104151
• Fixed a bug where if a user was logged in while a different session dropped that user, the dropped user would still inherit privileges from the public role. Now, CockroachDB checks that the user exists before allowing it to inherit privileges from the public role. In addition, any active web sessions are now revoked when a user is dropped. #104215
• Fixed a bug in upstream etcd-io/raft which could result in pulling unlimited amount of logs into memory, and lead to out-of-memory errors. Now the log scan has a limited memory footprint. #104483
• Fixed a bug where, in rare circumstances, a replication could get stuck when proposed near lease or leadership changes, especially under overload, and the [replica circuit breakers](../v23.2 could trip. A previous attempt to fix this issue has been reverted in favor of this fix. #106515
• CockroachDB now automatically deletes statistics for dropped tables from the system.table_statistics table. #105364
• Fixed a rare internal error which occurs when a query uses a "project set" operation involving simple column expressions. #104756
• The Raft PreVote and CheckQuorum mechanisms are now fully enabled. These prevent spurious elections when followers already have an active leader, and cause leaders to step down if they don't hear back from a quorum of followers. This improves reliability under partial and asymmetric network partitions, by avoiding spurious elections and preventing unavailability where a partially partitioned node could steal leadership away from an established leaseholder who would then no longer be able to reach the leader and submit writes. #104042
• Fixed a bug that could produce incorrect values for virtual computed columns in rare cases. The bug only occurred when the virtual column expression's type did not match the type of the virtual column. #105736
• Fixed a rounding error that could cause distributed execution for some decimal aggregate functions to return slightly inaccurate results in rare cases. #105694
• Fixed the StatementStatistics.Nodes to contain all of the nodes involved in the query. Fixed the region info in EXPLAIN ANALYZE (DISTSQL) for virtual clusters. #106587
• Fixed a bug that caused backups to fail if there are tables and functions of the same name. #106626
• Fixed edge cases in decimal and float evaluation for division operators. 'NaN'::DECIMAL / 0 will now return NaN instead of a division-by-zero error, and 0 / 'inf'::DECIMAL will return 0 instead of 0E-2019. #106472
• Fixed a bug present since before v22.2 that could cause a query with LIMIT and ORDER BY to return results in the wrong order. This bug could cause incorrect results as well if the LIMIT was nested within an outer query (e.g., under another LIMIT). #106717
• Added missing SQLInstanceIDs used to execute the statement to the telemetry SampledQuery event. #106753
• Fixed a bug where inserting geometries into a table with an inverted index involving a NaN coordinate could result in a panic. This now produces errors instead. #106671
• Avoid displaying undefined regions on the Databases page. #106778
• The cockroach userfile upload command uses less memory when uploading a file. #106056
• CASE, IF, COALESCE, and IFNULL expressions now return an error when passed a generator function as an argument. This mirrors the behavior of PostgreSQL. #105582
• Fixed a bug that allowed views created with CREATE OR REPLACE VIEW to reference user-defined types in other databases, even with sql.cross_db_views.enabled set to false. This bug was present since user-defined types were introduced in v20.1. #106869
• Removed a source of unnecessary Raft snapshots during replica movement. #106793
• Fixed a bug where in rare situations nodes would get stuck during start-up. It would manifest itself through a stack frame sitting on a select in waitForAdditionalStoreInit for extended periods of time (i.e., minutes). #107124
• Fixed a bug that caused internal errors when using an aggregate function in an ORDER BY clause of a DELETE or UPDATE statement. Aggregate functions are no longer allowed in these contexts. The bug has been present since at least v20.2. #107641
• The filter on the Statements page works when application name is an empty string. #107750
• The Transaction Details page now loads with the fingerprint details even if no application is specified in the URL. #107742
• The Schema Insights page no longer times out. #107292
• The last SQL statement in a user-defined function with a VOID return type can now produce any number of columns of any type. This bug was present since UDFs were introduced in v22.2. #108299
• Fixed a bug that caused nodes to crash when attempting to EXECUTE a prepared statement with an argument that referenced a user-defined function. This bug was present since user-defined functions were introduced in v22.2. #108213
• Fixed a bug where a release save point could incorrectly emit a "cannot publish new versions for descriptors" error instead of a retryable error. #108133
• Users with the VIEWACTIVITY privilege now are able to see other users sessions from both the CLI and the DB Console. #106590
• Fixed a bug in cockroach demo whereby \demo add could sometimes crash with an error "index out of range [...] with length ...". This bug had been introduced in v19.x. #108566
• Fixed a bug introduced in v20.2 where the command \demo decommission in cockroach demo could leave the demo cluster in a broken state. #108566
• Fixed a bug where cockroach start would sometimes incorrectly hang upon shutting down a server after encountering an internal error. This bug had been introduced some time in v22.x. #108612
• Fixed a bug in the index recommendations provided in the EXPLAIN output where ALTER INDEX ... VISIBLE index recommendations may suggest making the wrong index visible when there are multiple invisible indexes in a table. #108576
• Users with the VIEWACTIVITY privilege can now view correct values for timezones. #108486
• Fixed a bug present since v23.1.0 that would cause queries on the pg_catalog.pg_statistic_ext table to fail if a table was dropped recently. This bug also caused the \d CLI shortcut to encounter errors. #108818
• Fixed a bug where pg_attribute and pg_attrdef did not properly return results for generated columns. #108964
• Fixed a bug where a SpanStatsRequest would return post-replicated MVCC stats. Now, a SpanStatsRequest returns the logical MVCC stats for the requested span. #108852
• Fixed the column name on the selects on the tables crdb_internal.node_txn_execution_insights and crdb_internal.cluster_txn_execution_insights upon the creation of debug.zip. #109444
• Fixed the type resolution logic for CASE statements to more closely match Postgres' logic. In particular, we now adhere to rule 5 listed in the PostgreSQL documentation, which requires that we select the first non-unknown input type as the candidate type, then consider each other non-unknown input type, left to right (CASE treats its ELSE clause (if any) as the "first" input, with the THEN clauses(s) considered after that). If the candidate type can be implicitly converted to the other type, but not vice-versa, select the other type as the new candidate type. Then continue considering the remaining inputs. If, at any stage of this process, a preferred type is selected, stop considering additional inputs (note that CockroachDB does not yet support the concept of a "preferred type"). #108387
• Fixed an issue on the Metrics page where no metrics would load when viewing metrics for a virtual cluster with a hyphenated name in a global context. #109174
• Fixed a potential livelock between a high-priority transactional read and a normal-priority write. The read pushes the timestamp of the write, but if the read gets pushed as well, it may repeatedly fail to refresh because it keeps encountering the intent of the write. #108190
• Fixed a nil dereference panic during node startup that could be caused by an incorrect initialization order. #109659
• The difference built-in had its return type incorrectly set to a string instead of an integer. #109731
• Fixed a bug that could cause a transaction performing multiple parallel foreign key checks to return a concurrent txn use detected error. #109510
• Fixed a bug causing performance regression when disabling sql.metrics.statement_details.enabled which caused execution stats to be collected for all queries instead of the default one percent. #109785
• Fixed a bug where certain SQL session variables meant to be hidden from introspection were showing up in information_schema.session_variables, which was incoherent with the handling in pg_catalog.pg_settings. #109872
• CockroachDB now properly handles RPC failures on writes using the parallel commit protocol that execute in parallel to the commit operation, avoiding incorrect retryable failures and transaction unexpectedly committed assertions by detecting when writes cannot be retried idempotently, instead returning an AmbiguousResultError. #107658
• Fixed a bug where dependencies on sequences from tables would be reported with the wrong value for the classid column in the pg_catalog.pg_depend table. #110144
• Two ALTER RANGE default CONFIGURE ZONE statements on the same line no longer displays an error. #109774
• Fixed a DB Console issue where the DROP_UNUSED index recommendations produced by the table details page produced an invalid DROP INDEX statement. #110429
• Removed buggy TTL descriptor repair. Previously, upgrading from v22.2.X to v23.1.9 incorrectly removed TTL storage parameters from tables (visible by running a SHOW CREATE TABLE <ttl-table>; statement) while attempting to repair table descriptors. This resulted in the node that attempted to run the TTL job crashing due to a panic caused by the missing TTL storage parameters. #110364
• cockroach debug pebble commands now work correctly with encrypted stores which don't use the default cockroach-data path without having to also pass --store. #110150
• Fixed a bug where CREATE INDEX for partial indexes could fail with ERROR: duplicate key value violates unique constraint if concurrent inserts happened simultaneously. #110216
• Observability pages no longer crash when they encounter zeros (e.g., a session with no memory allocated). #108752
• Removed the cluster setting kv.snapshot_recovery.max_rate:
  • In v23.2, this setting is disabled; it is a no-op. If you previously set kv.snapshot_recovery.max_rate on a cluster running v23.1 and upgraded to v23.2, the setting is ignored, and the kv.snapshot_rebalance.max_rate setting is used instead.
  • In v24.1 and later, this setting is removed entirely. If you had previously set kv.snapshot_recovery.max_rate prior to upgrade, it will be cleared, and any attempts to set it will fail with the error message: ERROR: unknown cluster setting 'kv.snapshot_recovery.max_rate'. #102596

• Fixed a bug in which a CREATE FUNCTION may produce a syntax error if the UDF body wrapped in tagged dollar quotes (e.g., $func$), contains two consecutive dollar signs $$. If the UDF body is known to contain dollar signs, then the caller should use tagged dollar quotes or single quotes when defining the UDF. For example:

   CREATE FUNCTION f(a STRING) RETURNS STRING LANGUAGE SQL AS $func$ SELECT concat('$$', a); $func$
  

  #101352

• CockroachDB now prevents setting max_range_size below the COCKROACH_MIN_RANGE_MAX_BYTES environment variable, which defaults to 64 MiB (half of the default minimum range size). #96725

• Fixed a bug that could occasionally cause schema change jobs, such as table or index drops, to appear stuck in state "waiting for MVCC GC" for much longer than expected. The fix only applies to future schema changes. To process existing stuck jobs, manually force-enqueue the relevant ranges in the MVCC GC queue from the DB Console's Advanced Debug page. #110078

• Fixed a bug introduced when the ChartCatalog API endpoint was introduced, where the endpoint did not correctly report the unit of metrics. #109042

• Fixed a bug that could occur when the "multiple active portals" execution mode (Preview) was enabled to evaluate queries such as lookup joins. The bug could result in an internal error like unexpected 40960 leftover bytes if the portal was not fully consumed. #110625

• Fixed a bug where an ALTER TABLE ... ADD CONSTRAINT CHECK ... statement that utilized a user-defined function in the CHECK could cause a validation error. #110130

• Fixed a bug where RESET (ttl_expire_after) could incorrectly remove ttl_expiration_expression. #110252

• Fixed a bug where the format_type built-in did not honor typemod information for array types, leading to incorrect output. #110900

• Fixed a bug introduced in v22.2 that incorrectly allowed users without the EXECUTE privilege to execute a user-defined function. #107587

Performance improvements

• The optimizer now plans inverted index scans for queries using IN or the = operators without the fetch val (->) operator. For example: json_col = '{"b":"c"}' OR json_col IN ('"a"', '1') #101178
• Queries that have subqueries in equality expressions are now more efficiently planned by the optimizer. #100881
• Query planning time has been reduced for some queries with multiple joins. #102011
• CockroachDB now enables the pacing mechanism in rangefeed closed timestamp notifications, by setting the default kv.rangefeed.closed_timestamp_smear_interval cluster setting to 1ms. This makes rangefeed closed timestamp delivery more uniform and less spikey, which reduces its impact on the Go scheduler and, ultimately, foreground SQL latencies. #103006
• Some large, long-running INSERT statements now perform less work during their commit phase and can run faster. #103241
• Ranges now only quiesce after 3 seconds without proposals, to avoid frequent unquiescence which incurs an additional Raft proposal. This is configurable via the COCKROACH_QUIESCE_AFTER_TICKS environment variable, which defaults to 6. #103266
• SQL statements that must clean up intents from many different previously abandoned transactions now do so moderately more efficiently. #103265
• The optimizer can now avoid a grouping stage in more cases when de-duplicating the input to an UPSERT or INSERT ... ON CONFLICT statement. #105206
• The optimizer can now eliminate joins in more cases. #105214
• CockroachDB now improves the time to disk space reclamation when deleting rows. Previously, in scenarios where rows had large variations in row size, it was possible for disk space to not be reclaimed after MVCC garbage collection deleted the rows. #104539
• CockroachDB now has improved disk space reclamation heuristics, making disk space reclamation more timely. #106177
• bool_and and bool_or aggregates will now scale linearly instead of quadratically when used as a window function with a non-shrinking window, #106477
• CockroachDB now has reduced lock contention on ssmemstorage.RecordStatement. This is useful for workloads that execute the same statement concurrently on the same SQL instance. #106860
• The optimizer now produces more efficient query plans in some cases for queries with subqueries and user-defined functions. #107133
• The default Raft entry cache size has been increased from 16 MB to 1/256 of system memory with a minimum of 32 MB, divided evenly between all stores. This can be configured using the COCKROACH_RAFT_ENTRY_CACHE_SIZE environment variable. #107424
• CockroachDB now automatically collects table statistics on the system.jobs table, which will enable the optimizer to produce better query plans for internal queries that access the system.jobs table. This may result in better performance of the system. #108139
• The impact of high concurrency blind writes to the same key on goroutine scheduling latency was reduced. #109349
• Changefeeds to Webhook or Pub/Sub endpoints now support much higher throughput #109351
• This release improved the cost of resolving a user-defined enum type that has many values. #109394
• Queries that compare collated strings now use less memory and may execute faster. #110066
• Added a scheduler based rangefeed processor which improves rangefeed and changefeed performance for very large tables. The new processor is disabled by default, but can be enabled by setting kv.rangefeed.scheduler.enabled cluster setting to true. #107553
• This release disables sql.defaults.zigzag_join.enabled by default. #110214

Build changes

• Go has been upgraded to 1.20.8. #109773
• The top-level Makefile was replaced by a stub GNUmakefile which defers its behavior to dev. The common targets make [all], make test, and make install remain for compatibility with most UNIX installation guides. The previous make rules remain available via make -C build/GNUmakefile.obsolete. #84565

Contributors

This release includes 3208 merged PRs by 124 authors.