Node.js Application Developer’s Guide

MarkLogic Server

Node.js Application Developer's Guide

1

Copyright ? 2022 MarkLogic Corporation. All rights reserved.

MarkLogic 10 June, 2019

Last Revised: 10.0-9, February, 2022

MarkLogic Server

Table of Contents

Table of Contents

Node.js Application Developer's Guide

1.0 Introduction to the Node.js Client API ..........................................................9

1.1 Getting Started ........................................................................................................9 1.2 Required Software ................................................................................................14 1.3 Security Requirements ..........................................................................................15

1.3.1 Basic Security Requirements ....................................................................15 1.3.2 Controlling Document Access ..................................................................16 1.3.3 Evaluating Requests Against a Different Database ..................................16 1.3.4 Evaluating or Invoking Server-Side Code ................................................16 1.4 Terms and Definitions ..........................................................................................17 1.5 Key Concepts and Conventions ............................................................................18 1.5.1 MarkLogic Namespace .............................................................................18 1.5.2 Parameter Passing Conventions ................................................................18 1.5.3 Document Descriptor ................................................................................19 1.5.4 Supported Result Handling Techniques ...................................................19 1.5.5 Promise Result Handling Pattern ..............................................................20 1.5.6 Stream Result Handling Pattern ................................................................21 1.5.7 Streaming Into the Database .....................................................................22 1.5.8 Performing Point-in-Time Operations ......................................................23 1.5.9 Error Handling ..........................................................................................24 1.6 Creating a Database Client ...................................................................................25 1.7 Authentication and Connection Security ..............................................................26 1.7.1 Connecting to MarkLogic with SSL .........................................................27 1.7.2 Using SAML Authentication ....................................................................27 1.7.3 Using Certificate-Based Authentication ...................................................28

1.7.3.1 Obtaining a Client Certificate ...................................................28 1.7.3.2 Configuring Your App Server ...................................................29 1.7.3.3 Examples: Database Client Configuration ................................29 1.7.4 Using Kerberos Authentication ................................................................30 1.7.4.1 Configuring MarkLogic to Use Kerberos .................................30 1.7.4.2 Configuring Your Client Host for Kerberos .............................31 1.7.4.3 Creating a Database Client That Uses Kerberos .......................31 1.8 Using the Examples in This Guide .......................................................................31

2.0 Manipulating Documents .............................................................................33

2.1 Introduction to Document Operations ..................................................................33 2.2 Loading Documents into the Database .................................................................36

2.2.1 Overview ...................................................................................................36 2.2.2 Input Document Descriptors .....................................................................37

MarkLogic 10--June, 2019

Node.js Application Developer's Guide--Page 2

MarkLogic Server

Table of Contents

2.2.3 Calling Convention ...................................................................................38 2.2.4 Example: Loading A Single Document ....................................................39 2.2.5 Example: Loading Multiple Documents ...................................................40 2.2.6 Inserting or Updating Metadata for One Document .................................42 2.2.7 Automatically Generating Document URIs ..............................................43 2.2.8 Transforming Content During Ingestion ...................................................43 2.3 Reading Documents from the Database ................................................................44 2.3.1 Retrieving the Contents of a Document By URI ......................................45 2.3.2 Retrieving Metadata About a Document ..................................................46 2.3.3 Example: Retrieving Content and Metadata .............................................48 2.3.4 Transforming Content During Retrieval ...................................................50 2.4 Removing Content from the Database ..................................................................51 2.4.1 Removing Documents By URI .................................................................51 2.4.2 Removing Sets of Documents ...................................................................52 2.4.3 Removing All Documents ........................................................................53 2.5 Managing Collections of Objects and Documents ...............................................54 2.6 Performing a Lightweight Document Check ........................................................56 2.7 Conditional Updates Using Optimistic Locking ...................................................57 2.7.1 Understanding Optimistic Locking ...........................................................57 2.7.2 Enable Optimistic Locking .......................................................................58 2.7.3 Obtain a Version Id ...................................................................................59 2.7.4 Apply a Conditional Update .....................................................................60 2.8 Working with Binary Documents .........................................................................61 2.8.1 Type of Binary Documents .......................................................................61 2.8.2 Streaming Binary Content ........................................................................62 2.8.3 Retrieving Binary Content with Range Requests .....................................62 2.9 Working with Temporal Documents ....................................................................63 2.10 Working with Metadata ........................................................................................64 2.10.1 Metadata Categories .................................................................................64 2.10.2 Metadata Format .......................................................................................65 2.10.3 Working with Document Properties .........................................................67 2.10.4 Disabling Metadata Merging ....................................................................68

2.10.4.1 When to Consider Disabling Metadata Merging .......................68 2.10.4.2 How to Disable Metadata Merging ...........................................69

3.0 Patching Document Content or Metadata ....................................................70

3.1 Introduction to Content and Metadata Patching ...................................................70 3.2 Example: Adding a JSON Property ......................................................................72 3.3 Patch Reference ....................................................................................................73

3.3.1 insert ..........................................................................................................75 3.3.2 replace .......................................................................................................76 3.3.3 replaceInsert ..............................................................................................78 3.3.4 remove ......................................................................................................80 3.3.5 apply ..........................................................................................................81 3.3.6 library ........................................................................................................82 3.3.7 pathLanguage ............................................................................................82

MarkLogic 10--June, 2019

Node.js Application Developer's Guide--Page 3

MarkLogic Server

Table of Contents

3.3.8 collections .................................................................................................82 3.3.9 permissions ...............................................................................................83 3.3.10 properties ..................................................................................................83 3.3.11 quality .......................................................................................................83 3.3.12 metadataValues .........................................................................................83 3.4 Defining the Context for a Patch Operation .........................................................84 3.5 How Position Affects the Insertion Point .............................................................84 3.6 Patch Examples .....................................................................................................86 3.6.1 Preparing to Run the Examples ................................................................86 3.6.2 Example: Insert .........................................................................................87 3.6.3 Example: Replace .....................................................................................90 3.6.4 Example: ReplaceInsert ............................................................................93 3.6.5 Example: Remove .....................................................................................96 3.6.6 Example: Patching Metadata ....................................................................99 3.7 Creating a Patch Without a Builder ....................................................................102 3.8 Patching XML Documents .................................................................................103 3.9 Constructing Replacement Data on MarkLogic Server ......................................104 3.9.1 Overview of Replacement Constructor Functions ..................................105 3.9.2 Using a Builtin Replacement Constructor ..............................................106 3.9.3 Passing Parameters to a Replacement Constructor .................................107 3.9.4 Using a Custom Replacement Constructor .............................................107 3.9.5 Writing a Custom Replacement Constructor ..........................................108 3.9.6 Installing or Updating a Custom Replace Library ..................................109 3.9.7 Uninstalling a Custom Replace Library ..................................................110 3.9.8 Example: Custom Replacement Constructors ........................................111 3.9.9 Additional Operations .............................................................................116

4.0 Querying Documents and Metadata ...........................................................117

4.1 Query Interface Overview ..................................................................................117 4.2 Introduction to Search Concepts .........................................................................118

4.2.1 Search Overview .....................................................................................118 4.2.2 Query Styles ............................................................................................119 4.2.3 Types of Query .......................................................................................120 4.2.4 Indexing ..................................................................................................122 4.3 Understanding the queryBuilder Interface ..........................................................122 4.4 Searching with String Queries ............................................................................125 4.4.1 Introduction to String Query ...................................................................125 4.4.2 Example: Basic String Query .................................................................126 4.4.3 Using Constraints in a String Query .......................................................128 4.4.4 Example: Using Constraints in a String Query .......................................129 4.4.5 Using a Custom Constraint Parser ..........................................................131 4.4.6 Example: Custom Constraint Parser .......................................................132

4.4.6.1 Implementing the Constraint Parser ........................................132 4.4.6.2 Installing the Constraint Parser ...............................................133 4.4.6.3 Using the Custom Constraint in a String Query ......................133 4.4.7 Additional Information ...........................................................................135

MarkLogic 10--June, 2019

Node.js Application Developer's Guide--Page 4

MarkLogic Server

Table of Contents

4.5 Searching with Query By Example ....................................................................135 4.5.1 Introduction to QBE ................................................................................135 4.5.2 Creating a QBE with queryBuilder .........................................................136 4.5.3 Querying XML Content With QBE ........................................................138 4.5.4 Additional Information ...........................................................................139

4.6 Searching with Structured Queries .....................................................................140 4.6.1 Basic Usage .............................................................................................140 4.6.2 Example: Using Structured Query ..........................................................140 4.6.3 Builder Methods Taxonomy Reference ..................................................142 4.6.3.1 Basic Content Queries .............................................................143 4.6.3.2 Logical Composers ..................................................................145 4.6.3.3 Location Qualifiers ..................................................................145 4.6.3.4 Document Selectors .................................................................147 4.6.4 Query Parameter Helper Functions .........................................................147 4.6.5 Search Result Refiners ............................................................................149

4.7 Searching with Combined Query ........................................................................150 4.8 Searching Values Metadata Fields ......................................................................152 4.9 Querying Lexicons and Range Indexes ..............................................................152

4.9.1 Querying Values in a Lexicon or Range Index .......................................153 4.9.2 Finding Value Co-Occurrences in Lexicons ...........................................155 4.9.3 Building an Index Reference ..................................................................157 4.9.4 Refining the Results of a Values or Co-Occurrence Query ....................158 4.9.5 Analyzing Lexicons and Range Indexes with Aggregate Functions ......159

4.9.5.1 Aggregate Function Overview ................................................159 4.9.5.2 Using Builtin Aggregate Functions .........................................159 4.9.5.3 Using User-Defined Aggregate Functions ..............................160 4.10 Generating Search Facets ....................................................................................161 4.10.1 Defining a Simple Facet .........................................................................161 4.10.2 Naming a Facet .......................................................................................163 4.10.3 Including Facet Options ..........................................................................163 4.10.4 Defining Bucket Ranges .........................................................................163 4.10.5 Creating and Using Custom Constraint Facets .......................................164 4.11 Refining Query Results .......................................................................................165 4.11.1 Available Refinements ............................................................................165 4.11.2 Paginating Query Results ........................................................................166 4.11.3 Returning Metadata .................................................................................167 4.11.4 Excluding Document Descriptors or Values From Search Results ........167 4.11.5 Generating Search Snippets ....................................................................168 4.11.6 Transforming the Search Results ............................................................169 4.11.7 Extracting a Portion of Each Matching Document .................................170 4.12 Generating Search Term Completion Suggestions .............................................173 4.12.1 Understanding the Suggestion Interface .................................................173 4.12.2 Example: Generating Search Term Suggestions ....................................176 4.13 Loading the Example Data .................................................................................179

5.0 Using the Optic API for Relational Operations .........................................183

MarkLogic 10--June, 2019

Node.js Application Developer's Guide--Page 5

MarkLogic Server

Table of Contents

5.1 Introduction to the Optic Interfaces ....................................................................183 5.2 Interface Summary ..............................................................................................184 5.3 Preparing to Run the Examples ..........................................................................184 5.4 Generating a Plan ................................................................................................185 5.5 Invoking a Plan ...................................................................................................186 5.6 Configuring Row Set Format ..............................................................................189

5.6.1 Configuration Options ............................................................................189 5.6.2 Layout Examples ....................................................................................189 5.7 Streaming Row Data ...........................................................................................193 5.7.1 Object Mode Streaming ..........................................................................193 5.7.2 Chunked Mode Streaming ......................................................................195 5.7.3 Sequence Mode Streaming .....................................................................195 5.8 Passing Parameters into a Plan ...........................................................................197 5.9 Handling Complex Column Values ....................................................................197 5.10 Generating an Execution Plan .............................................................................198 5.11 Serializing a Plan ................................................................................................199

6.0 Working With Semantic Data ....................................................................201

6.1 Overview of Common Semantics Tasks .............................................................201 6.2 Loading Triples ...................................................................................................202 6.3 Querying Semantic Triples With SPARQL ........................................................204 6.4 Example: SPARQL Query ..................................................................................205 6.5 Managing Graphs ................................................................................................206

6.5.1 Creating or Replacing a Graph ...............................................................207 6.5.2 Adding Triples to an Existing Graph ......................................................207 6.5.3 Removing a Graph ..................................................................................208 6.5.4 Retrieving the Contents, Metadata, or Permissions of a Graph ..............209 6.5.5 Testing for Graph Existence ...................................................................210 6.5.6 Retrieving a List of Graphs .....................................................................211 6.6 Using SPARQL Update to Manage Graphs and Graph Data .............................211 6.7 Applying Inferencing Rules to a SPARQL Query or Update .............................213 6.7.1 Basic Inference Ruleset Usage ...............................................................213 6.7.2 Example: SPARQL Query With Inference Ruleset ................................214 6.7.3 Example: SPARQL Update With Inference Rulesets .............................214 6.7.4 Controlling the Default Database Ruleset ..............................................214

7.0 Managing Transactions ..............................................................................216

7.1 Transaction Overview .........................................................................................216 7.2 Creating a Transaction ........................................................................................217 7.3 Associating a Transaction with an Operation .....................................................218 7.4 Committing a Transaction ..................................................................................219 7.5 Rolling Back a Transaction .................................................................................219 7.6 Example: Using Promises With a Multi-Statement Transaction ........................220 7.7 Checking Transaction Status ..............................................................................220 7.8 Managing Transactions When Using a Load Balancer ......................................220

MarkLogic 10--June, 2019

Node.js Application Developer's Guide--Page 6

MarkLogic Server

Table of Contents

8.0 Extensions, Transformations, and Server-Side Code Execution ...............223

8.1 Ways to Extend and Customize the API .............................................................223 8.2 Working with Resource Service Extensions .......................................................224

8.2.1 What is a Resource Service Extension? ..................................................224 8.2.2 Creating a Resource Service Extension ..................................................225 8.2.3 Installing a Resource Service Extension .................................................225 8.2.4 Using a Resource Service Extension ......................................................227 8.2.5 Example: Installing and Using a Resource Service Extension ...............228 8.2.6 Retrieving the Implementation of a Resource Service Extension ..........231 8.2.7 Discovering Resource Service Extensions .............................................231 8.2.8 Deleting Resource Service Extensions ...................................................232 8.3 Working with Content Transformations .............................................................233 8.3.1 What is a Content Transformation? ........................................................233 8.3.2 Creating a Transformation ......................................................................234 8.3.3 Installing a Transformation .....................................................................234 8.3.4 Using a Transformation ..........................................................................235 8.3.5 Example: Read, Write, and Query Transforms .......................................237

8.3.5.1 Install the Transforms ..............................................................237 8.3.5.2 Use the Write Transform .........................................................238 8.3.5.3 Use the Read Transform ..........................................................240 8.3.5.4 Use the Query Transform ........................................................241 8.3.5.5 Read Transform Source Code .................................................243 8.3.5.6 Write Transform Source Code ................................................244 8.3.5.7 Query Transform Source Code ...............................................245 8.3.6 Discovering Installed Transforms ...........................................................246 8.3.7 Deleting a Transformation ......................................................................246 8.4 Error Reporting in Extensions and Transformations ..........................................247 8.4.1 Example: Reporting Errors in JavaScript ...............................................247 8.4.2 Example: Reporting Errors in XQuery ...................................................249 8.5 Evaluating Ad-Hoc Code and Server-Side Modules ..........................................250 8.5.1 Required Privileges .................................................................................250 8.5.2 Evaluating a Ad-Hoc Query ...................................................................251 8.5.3 Invoking a Module Installed on MarkLogic Server ................................253 8.5.4 Interpreting the Results of Eval or Invoke ..............................................255 8.5.5 Specifying External Variable Values ......................................................256 8.6 Managing Assets in the Modules Database ........................................................257 8.6.1 Overview of Asset Management .............................................................258 8.6.2 Installing or Updating an Asset ..............................................................260 8.6.3 Referencing an Asset from Server-Side Code ........................................260 8.6.4 Removing an Asset .................................................................................261 8.6.5 Retrieving an Asset List ..........................................................................261 8.6.6 Retrieving an Asset .................................................................................262

9.0 Administering REST API Instances ..........................................................263

9.1 What Is a REST API Instance? ...........................................................................263

MarkLogic 10--June, 2019

Node.js Application Developer's Guide--Page 7

MarkLogic Server

Table of Contents

9.2 Creating an Instance ............................................................................................264 9.3 Configuring Instance Properties .........................................................................264 9.4 Retrieving Configuration Information ................................................................266 9.5 Removing an Instance .........................................................................................266

10.0

Creating Data Services and Developer Actions in Node.js .......................267

10.1 Node.js Annotations for Declarations .................................................................268 10.2 Using Gulp to Generate Models .........................................................................269 10.3 Generated Modules .............................................................................................270 10.4 Expected Pattern of Use ......................................................................................270

11.0

Data Movement in the Node.js API ...........................................................272

11.1 Concurrency and Large Data Sets in Node.js .....................................................272 11.1.1 Optimal Concurrency ..............................................................................272 11.1.2 Detection of Server Factors ....................................................................273 11.1.3 IO With Node.js Streams ........................................................................273 11.1.4 Data Movement Functions ......................................................................274

11.2 Node-client-api - 2.8.0 ........................................................................................274 11.2.1 Ingesting Documents using - writeAll API .............................................274 11.2.1.1 writeAll (options) ....................................................................275

11.3 Node-client-api - 2.9.0 ........................................................................................279 11.3.1 Collecting Document uris - queryAll API ..............................................280 11.3.2 Exporting Documents - readAll API ......................................................284 11.3.2.1 queryToReadAll ......................................................................288

11.4 Node-client-api - 2.10.0 ......................................................................................289 11.4.1 Reprocessing Documents - transformAll api ..........................................289 11.4.1.1 queryToTransformAll .............................................................293 11.4.2 Deleting Documents - removeAllUris API .............................................294 11.4.2.1 queryToRemoveAll .................................................................298 11.4.3 Exporting Rows - queryAll API .............................................................298

12.0 Technical Support ......................................................................................305

13.0 Copyright ...................................................................................................307

MarkLogic 10--June, 2019

Node.js Application Developer's Guide--Page 8

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download