#redis

Redis is introducing a new data structure called Streams to address messaging and time series gaps. Streams support range queries, blocking operations, and capped collections with low memory usage. The feature will be available in Redis 4.0 before year-end.

A critical bug in Redis 4.0's PSYNC2 replication protocol caused slaves to fail processing writes from Lua scripts after restart. The issue required multiple patch releases (4.0.3 through 4.0.5) to fully resolve due to rushed fixes and testing gaps. The Redis maintainer acknowledged releasing version 4.0 too early and plans improvements to testing and release processes.

Redis Streams development is progressing with consumer groups implementation underway. The feature will be included in Redis 5.0, scheduled for release in about two months, rather than Redis 4.0. Streams are currently the main development priority with core functionality already merged into the unstable branch.

Redis developer antirez investigated crash reports involving radix tree memory access errors. The issue was traced to a read overflow bug that had already been fixed months earlier but wasn't properly backported. The investigation revealed valuable lessons about crash reporting, fuzz testing, and assembly-level debugging for system software.

A security report found that 75% of unprotected Redis servers exposed on the internet are infected with malware. Redis creator explains that these instances are misconfigured and not designed to be publicly accessible. The article discusses security improvements like protected mode and upcoming ACL features in Redis 6.

Redis has released security patches for multiple vulnerabilities in its Lua scripting subsystem, affecting the cmsgpack and struct libraries. These vulnerabilities could allow memory corruption and potential takeover of Redis processes, particularly in cloud hosting environments. The fixes are included in Redis versions 3.2.12, 4.0.10, and 5.0-rc2.

Redis remains BSD licensed, contrary to rumors about license changes. Certain Redis modules developed by Redis Labs are switching to Apache with Common Clause licensing, but the core Redis project continues under the permissive BSD license.

Redis creator clarifies that Redis remains BSD-licensed while Redis Labs develops proprietary modules for additional functionality. He argues this differs from "open core" models where core functionality is removed from free software. The modular approach allows Redis to stay focused while Redis Labs explores complementary features.

Redis creator Salvatore Sanfilippo explains his refusal to change the "master-slave" terminology in Redis replication. He argues that terminology out of context is not offensive and that political correctness focuses on formalities rather than substantive issues. Sanfilippo emphasizes his progressive political views while defending his position on terminology.

Redis creator antirez introduced LOLWUT, a new command that generates computer art inspired by Georg Nees' Schotter. The command uses Braille Unicode characters to create pixel art in terminal interfaces and will change with each major Redis version. LOLWUT serves as a reminder that programming originated from exploration, not just utility.

The author argues that code comments are essential for maintainable software, explaining why code does something rather than just what it does. He categorizes Redis comments into nine types including function, design, why, teacher, checklist, guide, trivial, debt, and backup comments, with examples from the Redis source code.

Redis 6 will only support the new RESP3 protocol, which introduces breaking changes but offers more semantic data types. The author explains mitigations including extended support for Redis 5 and a long testing period. Client libraries will need updates, but RESP3 allows automatic protocol detection for backward compatibility.

Redis creator Salvatore Sanfilippo addresses community concerns about RESP3 protocol, ACLs, and multi-threading. He explains that RESP3 aims to simplify client libraries while maintaining backward compatibility, ACLs focus on operational safety rather than enterprise security, and multi-threading will focus on slow commands rather than I/O scaling.

Redis creator Salvatore Sanfilippo released a Gopher protocol implementation for Redis to celebrate its 10th anniversary. The 100-line code addition allows Redis to serve Gopher content while maintaining normal Redis functionality. The project aims to provide an alternative, text-focused space outside the modern internet's chaos.

Redis Streams, introduced in Redis 5, are a general-purpose data structure beyond just messaging use cases. They function like time-ordered append-only logs with efficient range queries and compact memory usage. Streams can store structured data entries with unique IDs and serve various applications from time series to object modeling.

The author discusses the psychological challenges of maintaining a popular open source project like Redis, including overwhelming community feedback that shifts focus from creative work to issue management. Maintaining consistent work schedules and dealing with difficult community interactions add to the stress, despite the project's success and personal fulfillment.

Redis 6 introduces server-assisted client-side caching through a tracking feature. The system uses 24-bit CRC64 hashing to create over 16 million caching slots for fine-grained invalidation. Clients can opt into tracking and receive invalidation messages when cached keys are modified.

Redis 6 Release Candidate 1 has been released, featuring major new enterprise features including SSL encryption, ACLs for access control, and the RESP3 protocol. This is the largest Redis release to date with contributions from many developers, and includes client-side caching, threaded I/O, and improved modules APIs.

Redis 6.0.0 stable has been released with major features including SSL, ACLs, RESP3, client-side caching, and threaded I/O. The release includes redesigned client-side caching with broadcasting mode, better ACL logging, faster RDB loading, and new STRALGO command. It represents the biggest Redis release ever.

Redis creator Salvatore Sanfilippo (antirez) is stepping down as the project's maintainer after more than ten years. He will transition to an advisory role at Redis Labs while handing over maintenance responsibilities to Yossi Gottlieb and Oran Agra. Sanfilippo expressed that he wants to focus more on creative coding rather than project maintenance.

The author defends linked lists as valuable educational tools that teach fundamental computing concepts like time complexity. They highlight linked lists' versatility, noting their use in systems like Redis and Linux kernels, and their ability to be augmented into structures like skip lists.

Redis creator Salvatore Sanfilippo (antirez) is returning to the Redis ecosystem after a 4.5-year absence. He contacted Redis CEO Rowan Trollope about rejoining in a community-focused role and will work on bridging company-community relations while exploring new technical directions like vector search capabilities.

Vector Sets have been merged into Redis as a new core data type that stores vectors instead of scalar scores. The implementation supports vector similarity queries, true deletions, quantization, and filtering with JSON attributes. It uses a modified HNSW algorithm and threaded operations for performance.

The author reproduced a Hacker News writing style fingerprinting technique using Redis vector sets. They processed HN comment data to create style vectors based on word frequency z-scores, allowing detection of users with similar writing patterns. The method can identify potential duplicate accounts and distinguish between native and non-native English speakers.

Redis has returned to open source licensing under AGPLv3 after previously using SSPL. The author rejoined Redis five months ago and advocated for the license change, wanting his Vector Sets data type work to be open source. Redis 8 is now generally available as the first version with the new license.

The author reflects on lessons learned from switching Redis to the AGPL license. Key insights include the importance of license familiarity, community sentiment about software distributions, and differing generational perspectives on open source. The author also notes Redis and ValKey will continue to diverge, offering users distinct feature choices.

The author describes fixing a complex bug in Redis Vector Sets where corrupted data could cause use-after-free errors. While LLMs provided basic solutions, the author devised a more creative approach using XOR accumulators and hash functions to verify link reciprocity efficiently.

The author details scaling HNSWs for Redis Vector Sets, including 8-bit quantization for memory efficiency, threaded operations for performance, bidirectional linking for proper deletion, and exposing HNSWs as composable data structures. The implementation achieves 50k ops/sec with 3GB RAM for 3 million Word2Vec entries.

The Redis documentation provides comprehensive information about commands, data types, common patterns, configuration hints, and algorithms that can be implemented using Redis. The resource is available at redis.antirez.com and is useful for both LLMs/coding agents and human developers.