Best AI Image Generators

Best AI Image Generators — hands-on reviews, top picks, pricing, pros and cons and a practical how-to guide on Aizhi.

Computational photography

Computational photography refers to digital image capture and processing techniques that use digital computation instead of optical processes. Computational photography can improve the capabilities of a camera, or introduce features that were not possible at all with film-based photography, or reduce the cost or size of camera elements. Examples of computational photography include in-camera computation of digital panoramas, high-dynamic-range images, and light field cameras. Light field cameras use novel optical elements to capture three-dimensional scene information, which can then be used to produce 3D images, enhanced depth-of-field, and selective de-focusing (or "post focus"). Enhanced depth-of-field reduces the need for mechanical focusing systems. All of these features use computational imaging techniques. The definition of computational photography has evolved to cover a number of subject areas in computer graphics, computer vision, and applied optics. These areas are given below, organized according to a taxonomy proposed by Shree K. Nayar. Within each area is a list of techniques, and for each technique, one or two representative papers or books are cited. Deliberately omitted from the taxonomy are image processing (see also digital image processing) techniques applied to traditionally captured images to produce better images. Examples of such techniques are image scaling, dynamic range compression (i.e. tone mapping), color management, image completion (a.k.a. inpainting or hole filling), image compression, digital watermarking, and artistic image effects. Also omitted are techniques that produce range data, volume data, 3D models, 4D light fields, 4D, 6D, or 8D BRDFs, or other high-dimensional image-based representations. Epsilon photography is a sub-field of computational photography. == Effect on photography == Photos taken using computational photography can allow amateurs to produce photographs rivalling the quality of professional photographers, but as of 2019 do not outperform the use of professional-level equipment. == Computational illumination == This is controlling photographic illumination in a structured fashion, then processing the captured images, to create new images. The applications include image-based relighting, image enhancement, image deblurring, geometry/material recovery and so forth. High-dynamic-range imaging uses differently exposed pictures of the same scene to extend dynamic range. Other examples include processing and merging differently illuminated images of the same subject matter ("lightspace"). == Computational optics == This is a capture of optically coded images, followed by computational decoding to produce new images. Coded aperture imaging was mainly applied in astronomy and X-ray imaging to boost the image quality. Instead of a single pin-hole, a pinhole pattern is applied in imaging, and deconvolution is performed to recover the image. In coded exposure imaging, the on/off state of the shutter is coded to modify the kernel of motion blur. In this way, motion deblurring becomes a well-conditioned problem. Similarly, in a lens based coded aperture, the aperture can be modified by inserting a broadband mask. Thus, out of focus deblurring becomes a well-conditioned problem. The coded aperture can also improve the quality in light field acquisition using Hadamard transform optics. Coded aperture patterns can also be designed using color filters, in order to apply different codes at different wavelengths. This allows for increase the amount of light that reaches the camera sensor, compared to binary masks. == Computational imaging == Computational imaging is a set of imaging techniques that combine data acquisition and data processing to create the image of an object through indirect means to yield enhanced resolution, additional information such as optical phase or 3D reconstruction. The information is often recorded without using a conventional optical microscope configuration or with limited datasets. Computational imaging allows going beyond physical limitations of optical systems, such as numerical aperture, or even obliterates the need for optical elements. For parts of the optical spectrum where imaging elements such as objectives are difficult to manufacture or image sensors cannot be miniaturized, computational imaging provides useful alternatives, in fields such as X-ray and THz radiations. === Common techniques === Among common computational imaging techniques are lensless imaging, computational speckle imaging , ptychography and Fourier ptychography. Computational imaging technique often draws on compressive sensing or phase retrieval techniques, where the angular spectrum of the object is reconstructed. Other techniques are related to the field of computational imaging, such as digital holography, computer vision and inverse problems such as tomography. == Computational processing == This is the processing of non-optically-coded images to produce new images. == Computational sensors == These are detectors that combine sensing and processing, typically in hardware, like the oversampled binary image sensor. == Early work in computer vision == Although computational photography is a currently popular buzzword in computer graphics, many of its techniques first appeared in the computer vision literature, either under other names or within papers aimed at 3D shape analysis. == Art history == Computational photography, as an art form, has been practiced by capturing differently exposed pictures of the same subject matter and combining them. This was the inspiration for the development of the wearable computer in the 1970s and early 1980s. Computational photography was inspired by the work of Charles Wyckoff, and thus computational photography datasets (e.g. differently exposed pictures of the same subject matter that are taken in order to make a single composite image) are sometimes referred to as Wyckoff Sets, in his honor. Early work in this area (joint estimation of image projection and exposure value) was undertaken by Mann and Candoccia. Charles Wyckoff devoted much of his life to creating special kinds of 3-layer photographic films that captured different exposures of the same subject matter. A picture of a nuclear explosion, taken on Wyckoff's film, appeared on the cover of Life Magazine and showed the dynamic range from the dark outer areas to the inner core.
Read more →
List of information schools

This list of information schools, sometimes abbreviated to iSchools, includes members of the iSchools organization. The iSchools organization reflects a consortium of over 130 information schools across the globe. == History == The first iSchools Caucus was formed in 1988 by Syracuse, Pittsburgh, and Drexel and was called the Gang of Three (sometimes gang of four with Rutgers). Syracuse renamed the School of Library Science as the School of Information Studies in 1974, and is considered as the first “iSchool” in history. The group was formally named "the iSchools Caucus" or more casually, the iCaucus. By 2003, the group expanded to include the Universities of Michigan, Washington, Illinois, UNC, Florida State, Indiana, and Texas, and was called the Gang of Ten. The current iSchools Caucus organization was formalized by 2005, with additions of UC Berkeley, UC Irvine, UCLA, Penn State, Georgia Tech, Maryland, Toronto, Carnegie Mellon and Singapore Management University. == iSchools organization == The iSchools promote an interdisciplinary approach to understanding the opportunities and challenges of information management, with a core commitment to concepts like universal access and user-centered organization of information. The field is concerned broadly with questions of design and preservation across information spaces, from digital and virtual spaces such as online communities, social networking, the World Wide Web, and databases to physical spaces such as libraries, museums, collections, and other repositories. "School of Information", "Department of Information Studies", or "Information Department" are often the names of the participating organizations. Degree programs at iSchools include course offerings in areas such as information architecture, design, policy, and economics; knowledge management, user experience design, and usability; preservation and conservation; librarianship and library administration; the sociology of information; and human-computer interaction and computer science. === Leadership === The executive committee of the iSchools is made up of the current chair (Ina Fourie, University of Pretoria, South Africa), past chair (Gillian Oliver, Monash University, Australia) and the chair elect (Javed Mostafa, University of Toronto Canada), plus representatives from the three regions (North America, Europe, and Asia-Pacific). The current executive director is Slava Sterzer. == Member institutions == Between 2010 and 2026, the organization expanded globally beyond North America, growing to 133 member schools as of March 2026. For an updated and complete list of member schools, please visit the member database of the iSchools. == iConferences == Members of the iSchools organize a regular academic conference, known as the iConference, hosted by a different member institution each year. September 2005: Pennsylvania State University October 2006: University of Michigan February 2008: University of California, Los Angeles February 2009: University of North Carolina February 2010: University of Illinois at Urbana-Champaign February 2011: University of Washington, Seattle February 2012: University of Toronto February 2013: University of North Texas March 2014: Humboldt-Universität zu Berlin March 2015: University of California, Irvine March 2016: Drexel University March 2017: Wuhan University March 2018: University of Sheffield and Northumbria University March 2019: University of Maryland March 2020: University of Borås (virtual only) March 2021: Renmin University of China (virtual only) February/March 2022: University of Texas at Austin, University College Dublin & Kyushu University (virtual only) March 2023: Universitat Oberta de Catalunya March 2024: Jilin University March 2025: Indiana University March/April 2026: Edinburgh Napier University 2027: Victoria University of Wellington == Other schools of information == Other information schools and programs include: Documentation Research and Training Centre, Indian Statistical Institute, Bangalore San Jose State University, School of Information University of Southern California Library Science Degree Ankara University, Department of Information and Records Management, Ankara/Turkey Marmara University, Department of Information and Records Management, Istanbul/Turkey University of Kelaniya, Department of Library and Information Science, Kelaniya/Sri Lanka University of Colombo, National Institute of Library and Information Science (NILIS), Colombo/Sri Lanka Chicago State University, Department of Information Studies
Read more →
Microsoft Query

Microsoft Query is a visual method of creating database queries using examples based on a text string, the name of a document or a list of documents. The QBE system converts the user input into a formal database query using Structured Query Language (SQL) on the backend, allowing the user to perform powerful searches without having to explicitly compose them in SQL, and without even needing to know SQL. It is derived from Moshé M. Zloof's original Query by Example (QBE) implemented in the mid-1970s at IBM's Research Centre in Yorktown, New York. In the context of Microsoft Access, QBE is used for introducing students to database querying, and as a user-friendly database management system for small businesses. Microsoft Excel allows results of QBE queries to be embedded in spreadsheets.
Read more →
EJB QL

EJB QL or EJB-QL is a portable database query language for Enterprise Java Beans. It was used in Java EE applications. Compared to SQL, however, it is less complex but less powerful as well. == History == The language has been inspired, especially EJB3-QL, by the native Hibernate Query Language. In EJB3 It has been mostly replaced by the Java Persistence Query Language. == Differences == EJB QL is a database query language similar to SQL. The used queries are somewhat different from relational SQL, as it uses a so-called "abstract schema" of the enterprise beans instead of the relational model. In other words, EJB QL queries do not use tables and their components, but enterprise beans, their persistent state, and their relationships. The result of an SQL query is a set of rows with a fixed number of columns. The result of an EJB QL query is either a single object, a collection of entity objects of a given type, or a collection of values retrieved from CMP fields. One has to understand the data model of enterprise beans in order to write effective queries.
Read more →
Drop shadow

In graphic design and computer graphics, a drop shadow is a visual effect consisting of a drawing element which looks like the shadow of an object, giving the impression that the object is raised above the objects behind it. The drop shadow is often used for elements of a graphical user interface such as windows or menus, and for simple text. The text label for icons on desktops in many desktop environments has a drop shadow, as this effect effectively distinguishes the text from any colored background it may be in front of. A simple way of drawing a drop shadow of a rectangular object is to draw a gray or black area underneath and offset from the object. In general, a drop shadow is a copy in black or gray of the object, drawn in a slightly different position. Realism may be increased by: Darkening the colors of the pixels where the shadow casts instead of making them gray. This can be done with alpha blending the shadow with the area it is cast on. Softening the edges of the shadow. This can be done by adding Gaussian blur to the shadow's alpha channel before blending. Inset drop shadows are a type which draws the shadows inside the element. This allows the interface element to appear as if it is sunken into the interface. == Photo editing == In photo editing or photography post-production, a drop shadow may be added right beneath a model or product in the image. It is used to create contrast between the background and the subject. To add a drop shadow, retouchers use graphic editing tools like Adobe Photoshop. Drop shadows are often used as a visual effect in e-commerce. This is done to improve the presentation of product images and create depth in the image. == Use == Generally, window managers which are capable of compositing allow drop shadow effects, whereas incapable window managers do not. In some operating systems like macOS, drop shadow is used to differentiate between active and inactive windows. Websites are able to use drop shadow effects through the CSS properties box-shadow, text-shadow, and drop-shadow() filter function in filter. The first two are used for elements and text respectively, while the filter applies to the element's content, letting it support oddly shaped elements or transparent images.
Read more →
Evidence-based library and information practice

Evidence-based library and information practice (EBLIP) or evidence-based librarianship (EBL) is the use of evidence-based practices (EBP) in the field of library and information science (LIS). This means that all practical decisions made within LIS should 1) be based on research studies and 2) that these research studies are selected and interpreted according to some specific norms characteristic for EBP. Typically such norms disregard theoretical studies and qualitative studies and consider quantitative studies according to a narrow set of criteria of what counts as evidence. If such a narrow set of methodological criteria are not applied, it is better instead to speak of research based library and information practice. == Characteristics == Evidence-based practice in general has been characterised as a positivist approach; EBLIP is therefore also a positivist approach to LIS. As such, EBLIP is an approach in contrast to other approaches to LIS. The use of statistical approaches known as meta-analysis to conclude what evidence has been reported in the literature is one among other methods which is typical for the evidence-based approach. In 2002, Booth noted the three schools of EBILP had some commonalities, including the context of day-to-day decision-making, an emphasis on improving the quality of professional practice, a pragmatic focus on the 'best available evidence', incorporation of the user perspective, the acceptance of a broad range of quantitative and qualitative research designs, and access, either first-hand or second-hand, to the (process of) evidence-based practice and its products. He added one more, that EBILP is concerned with getting the best value for money. == The role of library and information science in EBP == Evidence-based practice in general is based on a very thorough search of the scientific literature and a very thorough selection and analysis of the retrieved literature. A close familiarity with database searching is needed, and library and information professionals have important roles to play in this respect. Therefore LIS professionals should be well suited to help professionals in other disciplines doing EBP. EBLIP is the application of this approach on LIS itself. It should be mentioned, however, that EBP started in medicine as evidence-based medicine (EBM) from which it spread to other fields. Only slowly and to a limited extent has EBP moved on to LIS. The EBLIP process can be applied to a variety of scenarios in LIS, including customer service, collection development, library management and information literacy instruction. In general, quantitative methods are used in LIS research. A 2010 study revealed five categories that capture the different ways library and information professionals experience evidence-based practice: Evidence-based practice is experienced as irrelevant; Evidence-based practice is experienced as learning from published research; Evidence-based practice is experienced as service improvement; Evidence-based practice is experienced as a way of being; Evidence-based practice is experienced as a weapon.
Read more →
Vector-field consistency

Vector-Field Consistency is a consistency model for replicated data (for example, objects), initially described in a paper which was awarded the best-paper prize in the ACM/IFIP/Usenix Middleware Conference 2007. It has since been enhanced for increased scalability and fault-tolerance in a recent paper. == Description == This consistency model was initially designed for replicated data management in ad hoc gaming in order to minimize bandwidth usage without sacrificing playability. Intuitively, it captures the notion that although players require, wish, and take advantage of information regarding the whole of the game world (as opposed to a restricted view to rooms, arenas, etc. of limited size employed in many multiplayer video games), they need to know information with greater freshness, frequency, and accuracy as other game entities are located closer and closer to the player's position. It prescribes a multidimensional divergence bounding scheme, based on a vector field that employs consistency vectors k=(θ,σ,ν), standing for maximum allowed time - or replica staleness, sequence - or missing updates, and value - or user-defined measured replica divergence, applied to all space coordinates in game scenario or world. The consistency vector-fields emanate from field-generators designated as pivots (for example, players) and field intensity attenuates as distance grows from these pivots in concentric or square-like regions. This consistency model unifies locality-awareness techniques employed in message routing and consistency enforcement for multiplayer games, with divergence bounding techniques traditionally employed in replicated database and web scenarios.
Read more →
Unrestricted algorithm

An unrestricted algorithm is an algorithm for the computation of a mathematical function that puts no restrictions on the range of the argument or on the precision that may be demanded in the result. The idea of such an algorithm was put forward by C. W. Clenshaw and F. W. J. Olver in a paper published in 1980. In the problem of developing algorithms for computing, as regards the values of a real-valued function of a real variable (e.g., g[x] in "restricted" algorithms), the error that can be tolerated in the result is specified in advance. An interval on the real line would also be specified for values when the values of a function are to be evaluated. Different algorithms may have to be applied for evaluating functions outside the interval. An unrestricted algorithm envisages a situation in which a user may stipulate the value of x and also the precision required in g(x) quite arbitrarily. The algorithm should then produce an acceptable result without failure.
Read more →
Progress in artificial intelligence

Progress in artificial intelligence (AI) refers to the advances, milestones, and breakthroughs that have been achieved in the field of artificial intelligence over time. AI is a branch of computer science that aims to create machines and systems capable of performing tasks that typically require human intelligence. AI applications have been used in a wide range of fields including medical diagnosis, finance, robotics, law, video games, agriculture, and scientific discovery. The society as a whole is looking for artificial intelligence to be on a key factor in the upcming years because of its potential. However, many AI applications are not perceived as AI: "A lot of cutting-edge AI has filtered into general applications, often without being called AI because once something becomes useful enough and common enough it's not labeled AI anymore." "Many thousands of AI applications are deeply embedded in the infrastructure of every industry." In the late 1990s and early 2000s, AI technology became widely used as elements of larger systems, but the field was rarely credited for these successes at the time. Kaplan and Haenlein structure artificial intelligence along three evolutionary stages: Artificial narrow intelligence – AI capable only of specific tasks; Artificial general intelligence – AI with ability in several areas, and able to autonomously solve problems they were never even designed for; Artificial superintelligence – AI capable of general tasks, including scientific creativity, social skills, and general wisdom. To allow comparison with human performance, artificial intelligence can be evaluated on constrained and well-defined problems. Such tests have been termed subject-matter expert Turing tests. Also, smaller problems provide more achievable goals and there are an ever-increasing number of positive results. In 2023, humans still substantially outperformed both GPT-4 and other models tested on the ConceptARC benchmark. Those models scored 60% on most, and 77% on one category, while humans scored 91% on all and 97% on one category. However, later research in 2025 showed that human-generated output grids were only accurate 73% of the time, while AI models available that year managed to score above 77%. == History == Increasing, promoting or constraining AI progress has often be done via controlling or increasing the amount of compute. == Current performance in specific areas == There are many useful abilities that can be described as showing some form of intelligence. This gives better insight into the comparative success of artificial intelligence in different areas. AI, like electricity or the steam engine, is a general-purpose technology. There is no consensus on how to characterize which tasks AI tends to excel at. Some versions of Moravec's paradox observe that humans are more likely to outperform machines in areas such as physical dexterity that have been the direct target of natural selection. While projects such as AlphaZero have succeeded in generating their own knowledge from scratch, many other machine learning projects require large training datasets. Researcher Andrew Ng has suggested, as a "highly imperfect rule of thumb", that "almost anything a typical human can do with less than one second of mental thought, we can probably now or in the near future automate using AI." Games provide a high-profile benchmark for assessing rates of progress; many games have a large professional player base and a well-established competitive rating system. AlphaGo brought the era of classical board-game benchmarks to a close when Artificial Intelligence proved their competitive edge over humans in 2016. Deep Mind's AlphaGo AI software program defeated the world's best professional Go Player Lee Sedol. Games of imperfect knowledge provide new challenges to AI in the area of game theory; the most prominent milestone in this area was brought to a close by Libratus' poker victory in 2017. E-sports continue to provide additional benchmarks; Facebook AI, Deepmind, and others have engaged with the popular StarCraft franchise of videogames. Broad classes of outcome for an AI test may be given as: optimal: it is not possible to perform better (note: some of these entries were solved by humans) super-human: performs better than all humans high-human: performs better than most humans par-human: performs similarly to most humans sub-human: performs worse than most humans === Optimal === Tic-tac-toe Connect Four: 1988 Checkers (aka 8x8 draughts): Weakly solved (2007) Rubik's Cube: Mostly solved (2010) Heads-up limit hold'em poker: Statistically optimal in the sense that "a human lifetime of play is not sufficient to establish with statistical significance that the strategy is not an exact solution" (2015) === Super-human === Othello (aka reversi): c. 1997 Scrabble: 2006 Backgammon: c. 1995–2002 Chess: Supercomputer (c. 1997); Personal computer (c. 2006); Mobile phone (c. 2009); Computer defeats human + computer (c. 2017) Jeopardy!: Question answering, although the machine did not use speech recognition (2011) Arimaa: 2015 Shogi: c. 2017 Go: 2017 Heads-up no-limit hold'em poker: 2017 Six-player no-limit hold'em poker: 2019 Gran Turismo Sport: 2022 === High-human === Crosswords: c. 2012 Freeciv: 2016 Dota 2: 2018 Bridge card-playing: According to a 2009 review, "the best programs are attaining expert status as (bridge) card players", excluding bidding. StarCraft II: 2019 Mahjong: 2019 Stratego: 2022 No-Press Diplomacy: 2022 Hanabi: 2022 Natural language processing === Par-human === Optical character recognition for ISO 1073-1:1976 and similar special characters. Classification of images Handwriting recognition Facial recognition Visual question answering SQuAD 2.0 English reading-comprehension benchmark (2019) SuperGLUE English-language understanding benchmark (2020) Some school science exams (2019) Some tasks based on Raven's Progressive Matrices Many Atari 2600 games (2015) === Sub-human === Optical character recognition for printed text (nearing par-human for Latin-script typewritten text) Object recognition Various robotics tasks that may require advances in robot hardware as well as AI, including: Stable bipedal locomotion: Bipedal robots can walk, but are less stable than human walkers (as of 2017) Humanoid soccer Speech recognition: "nearly equal to human performance" (2017) Explainability. Current medical systems can diagnose certain medical conditions well, but cannot explain to users why they made the diagnosis. Many tests of fluid intelligence (2020) Bongard visual cognition problems, such as the Bongard-LOGO benchmark (2020) Visual Commonsense Reasoning (VCR) benchmark (as of 2020) Stock market prediction: Financial data collection and processing using Machine Learning algorithms Angry Birds video game, as of 2020 Various tasks that are difficult to solve without contextual knowledge, including: Translation Word-sense disambiguation == Proposed tests of artificial intelligence == In his famous Turing test, Alan Turing picked language, the defining feature of human beings, for its basis. The Turing test is now considered too exploitable to be a meaningful benchmark. The Feigenbaum test, proposed by the inventor of expert systems, tests a machine's knowledge and expertise about a specific subject. A paper by Jim Gray of Microsoft in 2003 suggested extending the Turing test to speech understanding, speaking and recognizing objects and behavior. Proposed "universal intelligence" tests aim to compare how well machines, humans, and even non-human animals perform on problem sets that are generic as possible. At an extreme, the test suite can contain every possible problem, weighted by Kolmogorov complexity; however, these problem sets tend to be dominated by impoverished pattern-matching exercises where a tuned AI can easily exceed human performance levels. == Exams == According to OpenAI, in 2023 GPT-4 achieved high scores on several standardized and professional examinations, including around the 90th percentile on the Uniform Bar Exam, the 89th percentile on the mathematics section of the SAT, the 93rd percentile on SAT Reading and Writing, the 54th percentile on the analytical writing section of the GRE, the 88th percentile on GRE quantitative reasoning, and the 99th percentile on GRE verbal reasoning. OpenAI also reported that GPT-4 scored in the 99th to 100th percentile on the 2020 USA Biology Olympiad semifinal exam and earned top scores on several AP exams. Independent researchers found in 2023 that ChatGPT based on GPT-3.5 performed "at or near the passing threshold" on all three parts of the United States Medical Licensing Examination (USMLE), suggesting that large language models could reach passing-level performance on some medical knowledge assessments even without domain-specific fine-tuning. GPT-3.5 was also reported to attain a low but passing grade on examinations for four law school courses at the University of Minnes
Read more →
Ordered key–value store

An ordered key–value store (OKVS) is a type of data storage paradigm that can support multi-model databases. An OKVS is an ordered mapping of bytes to bytes. An OKVS will keep the key–value pairs sorted by the key lexicographic order. OKVS systems provides different set of features and performance trade-offs. Most of them are shipped as a library without network interfaces, in order to be embedded in another process. Most OKVS support ACID guarantees. Some OKVS are distributed databases. Ordered key–value stores found their way into many modern database systems including NewSQL database systems. == History == The origin of ordered key–value store stems from the work of Ken Thompson on dbm in 1979. Later in 1991, Berkeley DB was released that featured a B-Tree backend that allowed the keys to stay sorted. Berkeley DB was said to be very fast and made its way into various commercial product. It was included in Python standard library until 2.7. In 2009, Tokyo Cabinet was released that was superseded by Kyoto Cabinet that support both transaction and ordered keys. In 2011, LMDB was created to replace Berkeley DB in OpenLDAP. There is also Google's LevelDB that was forked by Facebook in 2012 as RocksDB. In 2014, WiredTiger, successor of Berkeley DB was acquired by MongoDB and is since 2019 the primary backend of MongoDB database. Other notable implementation of the OKVS paradigm are Sophia and SQLite3 LSM extension. Another notable use of OKVS paradigm is the multi-model database system called ArangoDB based on RocksDB. Some NewSQL databases are supported by ordered key–value stores. JanusGraph, a property graph database, has both a Berkeley DB backend and FoundationDB backend. == Key concepts == === Lexicographic encoding === There are algorithms that encode basic data types (boolean, string, number) and composition of those data types inside sorted containers (tuple, list, vector) that preserve their natural ordering. It is possible to work with an ordered key–value store without having to work directly with bytes. In FoundationDB, it is called the tuple layer. === Range query === Inside an OKVS, keys are ordered, and because of that it is possible to do range queries. A range query retrieves all keys between two specified keys, ensuring that the fetched keys are returned in a sorted order. === Subspaces === === Key composition === One can construct key spaces to build higher level abstractions. The idea is to construct keys, that takes advantage of the ordered nature of the top level key space. When taking advantage of the ordered nature of the key space, one can query ranges of keys that have particular pattern. === Denormalization === Denormalization, as in, repeating the same piece of data in multiple subspace is common practice. It allows to create secondary representation, also called indices, that will allow to speed up queries. == Higher level abstractions == The following abstraction or databases were built on top ordered key–value stores: Timeseries database, Record Database, also known as Row store databases, they behave similarly to what is dubbed RDBMS, Tuple Stores, also known as Triple Store or Quad Store but also Generic Tuple Store, Document database, that mimics MongoDB API, Full-text search Geographic Information Systems Property Graph Versioned Data Vector space database for Approximate Nearest Neighbor All those abstraction can co-exist with the same OKVS database and when ACID is supported, the operations happens with the guarantees offered by the transaction system. == Feature matrix == == Use-cases == OKVS are useful to implement two strategies: optimize a small feature e.g. to make a 10% improvement in read or write latency; the second strategy is to take advantage of the distributed nature of FoundationDB, and TiKV, for which there is no equivalent at very large scale in resilience. Both users need to re-implement the needed high level abstractions, because there are no portable ready-to-use libraries of high-level abstraction. There is still a complex balance, of complexity, maintainability, fine-tuning, and readily available features that makes it still a choice of experts. Sometime more specialized data-structures can be faster than a high-level abstraction on top of an OKVS. Another interest of OKVS paradigm stems from it simple, and versatile interface, that makes it an interesting target for experimental storage algorithms, and data structures.
Read more →
Navigational database

A navigational database is a type of database in which records or objects are found primarily by following references from other objects. The term was popularized by the title of Charles Bachman's 1973 Turing Award paper, The Programmer as Navigator. This paper emphasized the fact that the new disk-based database systems allowed the programmer to choose arbitrary navigational routes following relationships from record to record, contrasting this with the constraints of earlier magnetic-tape and punched card systems where data access was strictly sequential. One of the earliest navigational databases was Integrated Data Store (IDS), which was developed by Bachman for General Electric in the 1960s. IDS became the basis for the CODASYL database model in 1969. Although Bachman described the concept of navigation in abstract terms, the idea of navigational access came to be associated strongly with the procedural design of the CODASYL Data Manipulation Language. Writing in 1982, for example, Tsichritzis and Lochovsky state that "The notion of currency is central to the concept of navigation." By the notion of currency, they refer to the idea that a program maintains (explicitly or implicitly) a current position in any sequence of records that it is processing, and that operations such as GET NEXT and GET PRIOR retrieve records relative to this current position, while also changing the current position to the record that is retrieved. Navigational database programming thus came to be seen as intrinsically procedural; and moreover to depend on the maintenance of an implicit set of global variables (currency indicators) holding the current state. As such, the approach was seen as diametrically opposed to the declarative programming style used by the relational model. The declarative nature of relational languages such as SQL offered better programmer productivity and a higher level of data independence (that is, the ability of programs to continue working as the database structure evolves.) Navigational interfaces, as a result, were gradually eclipsed during the 1980s by declarative query languages. During the 1990s it started becoming clear that for certain applications handling complex data (for example, spatial databases and engineering databases), the relational calculus had limitations. At that time, a reappraisal of the entire database market began, with several companies describing the new systems using the marketing term NoSQL. Many of these systems introduced data manipulation languages which, while far removed from the CODASYL DML with its currency indicators, could be understood as implementing Bachman's "navigational" vision. Some of these languages are procedural; others (such as XPath) are entirely declarative. Offshoots of the navigational concept, such as the graph database, found new uses in modern transaction processing workloads. == Description == Navigational access is traditionally associated with the network model and hierarchical model of database, and conventionally describes data manipulation APIs in which records (or objects) are processed one at a time, iteratively. The essential characteristic as described by Bachman, however, is finding records by virtue of their relationship to other records: so an interface can still be navigational if it has set-oriented features. From this viewpoint, the key difference between navigational data manipulation languages and relational languages is the use of explicit named relationships rather than value-based joins: for department with name="Sales", find all employees in set department-employees versus find employees, departments where employee.department-code = department.code and department.name="Sales". In practice, however, most navigational APIs have been procedural: the above query would be executed using procedural logic along the lines of the following pseudo-code: On this viewpoint, the key difference between navigational APIs and the relational model (implemented in relational databases) is that relational APIs use "declarative" or logic programming techniques that ask the system what to fetch, while navigational APIs instruct the system in a sequence of steps how to reach the required records. Most criticisms of navigational APIs fall into one of two categories: Usability: application code quickly becomes unreadable and difficult to debug Data independence: application code needs to change whenever the data structure changes For many years the primary defence of navigational APIs was performance. Database systems that support navigational APIs often use internal storage structures that contain physical links or pointers from one record to another. While such structures may allow very efficient navigation, they have disadvantages because it becomes difficult to reorganize the physical placement of data. It is quite possible to implement navigational APIs without low-level pointer chasing (Bachman's paper envisaged logical relationships being implemented just as in relational systems, using primary keys and foreign keys), so the two ideas should not be conflated. But without the performance benefits of low-level pointers, navigational APIs become harder to justify. Hierarchical models often construct primary keys for records by concatenating the keys that appear at each level in the hierarchy. Such composite identifiers are found in computer file names (/usr/david/docs/index.txt), in URIs, in the Dewey decimal system, and for that matter in postal addresses. Such a composite key can be considered as representing a navigational path to a record; but equally, it can be considered as a simple primary key allowing associative access. As relational systems came to prominence in the 1980s, navigational APIs (and in particular, procedural APIs) were criticized and fell out of favour. The 1990s, however, brought a new wave of object-oriented databases that often provided both declarative and procedural interfaces. One explanation for this is that they were often used to represent graph-structured information (for example spatial data and engineering data) where access is inherently recursive: the mathematics originally underpinning SQL (specifically, first-order predicate calculus) does not have sufficient power to support recursive queries, even those as simple as a transitive closure. More recent SQL implementations do support hierarchical and recursive queries. A current example of a popular navigational API can be found in the Document Object Model (DOM) often used in web browsers and closely associated with JavaScript. The DOM is essentially an in-memory hierarchical database with an API that is both procedural and navigational. By contrast, the same data (XML or HTML) can be accessed using XPath, which can be categorized as declarative and navigational: data is accessed by following relationships, but the calling program does not issue a sequence of instructions to be followed in order. Languages such as SPARQL used to retrieve Linked Data from the Semantic Web are also simultaneously declarative and navigational. == Examples == IBM Information Management System IDMS
Read more →
Explore-then-commit algorithm

Explore Then Commit (ETC) is an algorithm for the multi-armed bandit problem foc,used on finding the best trade-off between exploration and exploitation. == Multi-armed bandit problem == The multi-armed bandit problem is a sequential game where one player has to choose at each turn between K {\displaystyle K} actions (arms). Behind every arm a {\displaystyle a} is an unknown distribution ν a {\displaystyle \nu _{a}} that lies in a set D {\displaystyle {\mathcal {D}}} known by the player (for example, D {\displaystyle {\mathcal {D}}} can be the set of Gaussian distributions or Bernoulli distributions). At each turn t {\displaystyle t} the player chooses (pulls) an arm a t {\displaystyle a_{t}} , they then get an observation X t {\displaystyle X_{t}} of the distribution ν a t {\displaystyle \nu _{a_{t}}} . === Regret minimization === The goal is to minimize the regret at time T {\displaystyle T} that is defined as R T := ∑ a = 1 K Δ a E [ N a ( T ) ] {\displaystyle R_{T}:=\sum _{a=1}^{K}\Delta _{a}\mathbb {E} [N_{a}(T)]} where μ a := E [ ν a ] {\displaystyle \mu _{a}:=\mathbb {E} [\nu _{a}]} is the mean of arm a {\displaystyle a} μ ∗ := max a μ a {\displaystyle \mu ^{}:=\max _{a}\mu _{a}} is the highest mean Δ a := μ ∗ − μ a {\displaystyle \Delta _{a}:=\mu ^{}-\mu _{a}} N a ( t ) {\displaystyle N_{a}(t)} is the number of pulls of arm a {\displaystyle a} up to turn t {\displaystyle t} The player has to find an algorithm that chooses at each turn t {\displaystyle t} which arm to pull based on the previous actions and observations ( a s , X s ) s < t {\displaystyle (a_{s},X_{s})_{s Read more →
Open Data Center Alliance

opendatacenteralliance.org appears to have been closed down. The Open Data Center Alliance is an independent organization created in Oct. 2010 with the assistance of Intel to coordinate the development of standards for cloud computing. Approximately 100 companies, which account for more than $50bn of IT spending, have joined the Alliance, including BMW, Royal Dutch Shell and Marriott Hotels. "The Alliance's Cloud 2015 vision is aimed at creating a federated cloud where common standards will be laid down for those in the hardware and software arena." == Usage Model Roadmap == The organization sees a growing need for solutions developed in an open, industry-standard and multivendor fashion, and has thus created a usage model roadmap featuring 19 prioritized usage models. The usage models provide detailed requirements for data center and cloud solutions, and will include detailed technical documentation discussing the requirements for technology deployments. To further its roadmap development, the steering committee established five initial technical workgroups in the areas of infrastructure, management, regulation & ecosystem, security and services. The organization delivered a 0.50 usage model roadmap to Open Data Center Alliance technical workgroups in Oct. 2010, and delivered a full 1.0 roadmap for public use in June 2011. == Membership == The steering committee consists of BMW, Capgemini, China Life, China Unicom Group, Deutsche Bank, JPMorgan Chase, Lockheed Martin, Marriott International, Inc., National Australia Bank, Royal Dutch Shell, Terremark and UBS. Other members include AT&T, CERN, eBay, Logica, Motorola Mobility Inc. and Nokia. "The demands on the IT organisations are coming at such an alarming rate that there are many, many different solutions being developed today that maybe don't work with each other. We need one voice, one road map, so that companies are able to say to manufacturers here is a clear vision of what they should be developing their product to do." says Marvin Wheeler, of Terremark, chairman of the Alliance. "While it's unclear how successful this alliance will be, it is at least shedding the spotlight on cloud interoperability, a big emerging issue," said Larry Dignan of ZDNet.
Read more →
Non-personal data

Non-Personal Data (NPD) is electronic data that does not contain any information that can be used to identify a natural person. Thus, it can either be data that has no personal information to begin with (such as weather data, stock prices, data from anonymous IoT sensors); or it is data that had personal data that was subsequently pseudoanonymized (for example, identifiable strings substituted with random strings) or anonymized (such as by irreversibly removing all personal data). NPD is part of the overall Data Governance Strategy of a region or country. While personal data are covered by Data Protection Legislation such as GDPR, other kinds of data would fall under the scope of NPD Regulation. == Importance of non-personal data == It has been pointed out that the future is data-driven. What this means is that much of the present innovation taking place in domains such as Machine Learning and Artificial Intelligence is fueled by data, which is needed for calibrating the complex models (comprising neural network-based as well as other kinds). The larger the volume, diversity and quality of the data, the higher is the quality of the model, leading to better predictions and explanations. However, there is a flip-side to data availability. The newly-emerging awareness of privacy and the consequent need for powerful Data Protection Regulations (such as GDPR) makes it increasingly difficult or impossible to obtain data in the quantities required. This is a contradiction, and the only way out would be to remove all personal data from data sets (either by Data anonymization or Pseudonymization coupled with noise injection, at which point it becomes NPD. Therefore, many innovation-friendly countries are coming out with regulatory regimes that would ensure that personal data is protected, while, at the same time, non-personal data can be extracted from personal data so that innovation is fostered. In other words, NPD 'unlocks' value that was locked away in data sets that have personally-identifiable information. It is expected that multiple NPD data sets will begin to be available on free or commercial basis from different providers once the regulations are in place. == Emerging regulatory frameworks == Non-Personal Data has significant uses that may be economic, social, political or security-related. Several countries and regions are in the process of regulating the use of NPD. In May 2019, the European Union operationalized its Regulation of the Free Flow of NPD. India announced a nine-member expert committee to make recommendations on the regulation of NPD in 2019, which published its first report in mid-2020. The report was opened for public comments, after which it was revised and published in December 2020. == Proposed NPD regulatory framework in India == The following were the objectives of the proposed Indian regulation as per the revised report: Sovereignty: India has rights over the data of India, its people and organisations. Benefit India: Benefits of data must accrue to India and its people. Benefits the world: Innovation, new models and algorithms for the world. Privacy: Misuse, reidentification and harms must be prevented. Simplicity: The regulations should be simple, digital and unambiguous. Innovation and entrepreneurship: The data should be freely available for innovation and entrepreneurship in India. == Concerns == The major concern in the use of NPD is if there are techniques (statistical or AI-based) by which multiple data sets can be used to extract personally-identifiable data.
Read more →
Virtual data room

A virtual data room (sometimes called a VDR or Deal Room) is an online repository of information that is used for the storing and distribution of documents. In many cases, a virtual data room is used to facilitate the due diligence process during an M&A transaction, loan syndication, or private equity and venture capital transactions. This due diligence process has traditionally used a physical data room to accomplish the disclosure of documents. For reasons of cost, efficiency and security, virtual data rooms have widely replaced the more traditional physical data room. A virtual data room is an extranet to which the bidders and their advisers are given access via the internet. An extranet is essentially a website with limited controlled access, using a secure log-on supplied by the vendor, which can be disabled at any time, by the vendor, if a bidder withdraws. Much of the information released is confidential and restrictions are applied to the viewer's ability to release this to third parties (by means of forwarding, copying or printing). This can be effectively applied to protect the data using digital rights management. The virtual data room provides access to secure documents for authorized users through a dedicated web site, or through secure agent applications. In the process of mergers and acquisitions the data room is set up as part of the central repository of data relating to companies or divisions being acquired or sold. The data room enables the interested parties to view information relating to the business in a controlled environment where confidentiality can be preserved. Conventionally this was achieved by establishing a supervised, physical data room in secure premises with controlled access. In most cases, with a physical data room, only one bidder team can access the room at a time. A virtual data room is designed to have the same advantages as a conventional data room (controlling access, viewing, copying and printing, etc.) with fewer disadvantages. Due to their increased efficiency, many businesses and industries have moved to using virtual data rooms instead of physical data rooms. In 2006, a spokesperson for a company which sets up virtual deal rooms was reported claiming that the process reduced the bidding process by about thirty days compared to physical data rooms. In the process of startup fundraising, a virtual data room is set up to be a central location for key data, documents, and financials. These are shared with venture capital and angel investors and allows them to streamline due diligence. == Application == Any business dealing with private data can apply VDRs when secure transaction processing is required. This includes financial institutions that need to negotiate confidential customer information without involving third parties. VDRs have traditionally been used for IPOs and real estate asset management. Technology companies may use them to exchange and review code or confidential data needed for operations. The same is true for clients, who entrust their valuable code only to the most qualified people in the organisation. The code is not something that can be printed out and brought in a folder. It resides on a computer and must be used together. VDR can find application in any business that manages data in the form of documents, especially law firms, financial advisers or the B2B sector. The latter work with documents that must always be handled and controlled confidentially, and it is difficult to store them securely when they are on a server that other people can access. In addition, in B2B, it is important to close the deal as quickly as possible: the average sales cycle is one to three months. VDR can be compared to a locked filing cabinet where all those folders and documents are kept. It automates the mathematics of pricing to prevent revenue leakage, and initially integrates CRM to ensure accurate synchronisation of all account data, which is important for B2B in particular and sales in general. While virtual data rooms offer many advantages, they are not suitable for every industry. For example, some governments may decide to continue using physical data rooms for highly confidential information sharing. The damage from potential cyberattacks and data breaches exceeds the benefits offered by virtual data rooms. In such cases, the use of VDRs is not considered. Data breaches have particularly affected the US healthcare system from March 2021 to March 2022 - according to IBM Security the cost of the breach was a record high of $10.1 million.
Read more →