Enterprise Framework

Software Solutions in the Enterprise

GraphQL - Dice Example without the GraphQL schema language:

GraphQL - Dice Example without the GraphQL schema language:

Using GraphQL schema language, you would use root and set it to the schema.



var express = require('express')
var graphqlHTTP = require('express-graphql')
var graphql = require('graphql')
const { GraphQLInt, GraphQLList, GraphQLObjectType, GraphQLString } = require('graphql')

class RandomDie {
  constructor (numSides) {
    this.numSides = numSides;
  }

  rollOnce () {
    return 1 + Math.floor(Math.random() * this.numSides);
  }

  roll ({numRolls}) {
    var output = [];
    for (var i = 0; i < numRolls; i++) {
      output.push(this.rollOnce())
    }
    return output
  }
}

// Define the Query type
var RandomDieType = new GraphQLObjectType({
  name: 'RandomDie',
  fields: {
    numSides: { type: GraphQLInt },
    rollOnce: { type: GraphQLInt },
    otherValue: { type: GraphQLString },
    roll: {
      type: new GraphQLList(graphql.GraphQLInt),
      args: {
        numRolls: { type: graphql.GraphQLInt }
      },
      resolve: function (_, {numRolls}, context) {
        // "_" represents the parent node in this case "getDie Type".  
        // It's the result of this.fields bound to the parent nodes
        // resolve data.
        // _ = {
        //   numSides: numSides,
        //   rollOnce: randomDie.rollOnce(),
        //   randomDie: randomDie
        // }
        console.log(_)
        console.log(`hello ${_.otherValue}`)
        var randomDie = new RandomDie(6)
        return randomDie.roll({
          numRolls: numRolls
        })
      }
    }
  }
})

var getDie = new graphql.GraphQLObjectType({
  name: 'Query',
  fields: {
    getDie: {
      type: RandomDieType,
      args: {
        numSides: { type: graphql.GraphQLInt }
      },
      resolve: function (_, {numSides}, context) {
        var randomDie = new RandomDie(numSides || 6)
        // THIS INFORMATION WILL BE PASSED TO "type: RandomDieType" RESOLVE
        // WITH THE FIELDS SET TO _
        return {
          numSides: numSides,
          rollOnce: randomDie.rollOnce(),
          randomDie: randomDie
        }
      }
    }
  }
})

var schema = new graphql.GraphQLSchema(
  {
    query: getDie
  }
)

var app = express()
app.use('/graphql', graphqlHTTP({
  schema: schema,
  rootValue: root,
  graphiql: true
}))
app.listen(4000)
console.log('Running a GraphQL API server at localhost:4000/graphql')

Request

{ getDie(numSides: 6) { numSides rollOnce roll(numRolls: 3) } }

Outputs

{ "data": { "getDie": { "numSides": 6, "rollOnce": 3, "roll": [ 1, 6, 3 ] } } }


Node JS - Standard JS -> Errors ('describe' is not defined, 'it' is not defined, 'jest' is not defined, 'expect' is not defined)

Running standard --fix on node shows errors ('describe' is not defined, 'it' is not defined, 'jest' is not defined, 'expect' is not defined)

$ standard --fix

standard: Use JavaScript Standard Style (https://standardjs.com)

  /test/getUser.test.js:7:1: 'jest' is not defined.

  /test/getUser.test.js:12:1: 'describe' is not defined.

  /test/getUser.test.js:13:3: 'it' is not defined.

  /test/getUser.test.js:15:5: 'expect' is not defined.

Add line to package.json
  "standard": {
    "env": [
      "jest"
    ]
  }


or add comment to the unit test file 

/* eslint-env mocha */

ElasticSearch - POST JSON to Remove hits .total, .max_score, ._index, ._type, _id, ._score

PROBLEM:  ElasticSearch - POST JSON to Remove hits .total, .max_score, ._index, ._type, _id, ._score

I have to be honest, this one really upset and frustrated me because I couldn't find an easy solution/answer using Google or ElasticSearch Documentation on how to only return back _source from a search POST json body.  When I finally figured it out, I think it wasn't implemented correctly. 

SOLUTION using QUERYSTRING:

First, off if you finally find in it in the ElasticSearch documentation, you can use the query string to remove extra metadata fields like _id, max_score, _index, _type, etc...  Correction, you can tell it which fields to include through a querystring:

GET /_search?q=elasticsearch&filter_path=took,hits.hits._id,hits.hits._score

reference:  https://www.elastic.co/guide/en/elasticsearch/reference/5.6/common-options.html

You can see from the example that you need to set the filter_path and pass in the fields you want returned back.

HOWEVER!!!!, if you are using the ElasticSearch-JS client for Node/browser, it doesn't really tell you how to do it in the API examples or in a POST json body example.

If this is documented, I sure as heck couldn't find it.

THE POST BODY SOLUTION

I'm going to tell you how to do it and there is a bug.

This is the normal client json request

            this.client.search({
                index: index,
                type: type,
                from: (page - 1) * pageSize,
                size: pageSize,
                body: body
            })

But what you can actually do it pass it filter_path in the search method options
            this.client.search({
                index: index,
                type: type,
                from: (page - 1) * pageSize,
                size: pageSize,
                body: body,
                filter_path: [
                    'hits.total,hits.max_score,hits.hits._id,hits.hits._score,hits.hits._source'                    
                ]
            })

But if you look at it, you're like, WTF? it's an array, however he's passing in a string delimited by comma(,)

Passing in the different values individually returns back weird results. For example, if i pass in

[ 'hits.total', 'hits.max_score' ]

On the results back, total will be undefined or max_score will be set.

By doing it all in a single string delimited by comma(,) it worked as expected. I should submit a bug but it's late, and i'm tired.

Hopefully, this helps somebody!!!






ElasticSearch - return back only specific fields in source

How to get ElasticSearch to return only specific source fields.  FYI, this is called Source Filtering on elastic.co's website.

Example: 1

in the request payload, specify 

       _source: ["fieldname1", "fieldname2"]


Example:

Sample Data

POST /library/books/_bulk

{ "index": { "_id": 1 } }

{ "title": "The quick brown fox", "price": 5, "colors": ["red","green","blue"] }

{ "index": { "_id": 2 } }

{ "title": "The quick brown fox jumps over the lazy dog", "price": 6, "colors": ["blue","yellow"] }

{ "index": { "_id": 3 } }

{ "title": "The quick brown fox jumps over the quick dog", "price": 7, "colors":  ["green", "purple"] }

{ "index": { "_id": 4 } }

{ "title": "The quick brown fox jumps over the stupid dog", "price": 8, "colors": ["green","yellow"] }

{ "index": { "_id": 5 } }

{ "title": "The quick brown fox jumped and farted", "price": 9, "colors": ["yellow", "purple"] }


Search with _source - return back only title and price

GET /library/books/_search

{

    "_source": [ "title", "price" ]

}


Example: 2

Sample Data


POST /library/books/6
{ 
  "title": "The quick brown fox", 
  "price": 5, 
  "colors": ["red","green","blue"],
  "contacts": [
    {
      "id": 1,
      "name": "john doe"
    },
    {
      "id": 2,
      "name": "john doe"
    },
    {
      "id": 3,
      "name": "john doe"
    }
  ]
}


Search with _source Example

GET /_search

{

    "_source": {

        "includes": [ "title", "price" ],

        "excludes": [ "colors" ]

    },

    "query" : {

        "term" : { "user" : "kimchy" }

    }

}


Reference Documentation:

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-source-filtering.html

GraphQL with ElasticSearch


Official Elastic.co ElasticSearch client for NodeJS, Browser client

$ npm install elasticsearch

Resources:


Get graphql-compose-elasticsearch

$ npm install graphql graphql-compose elasticsearch graphql-compose-elasticsearch --save

Mac Sierra : Install Yarn Errors

Yarn install issue on Mac

After trying to install Yarn from command line:

$ brew update

$ brew install yarn

Warning: Building gcc from source: 

The bottle needs the Xcode CLT to be installed. xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun Error: gcc cannot be built with any available compilers. Install GNU's GCC brew install gcc

After researching, I need to install the latest xcode command line tools.

$ xcode-select --install

Then I was able to run Yarn install

$ brew install yarn

ElasticSearch 5.6 Template To Apply Dynamic Template Mapping or Properties to Any Index and Type

Problem:  Have an ElasticSearch Dynamic Template that will apply Templates and/or new properties to any new Index or Type created.  

Anytime an Index is created with a new type it should

  • Have a default DynamicTemplate
  • Add the properties "timestamp" and "creation_date". 

First use the following resources:

https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-templates.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/default-mapping.html

That only gets you part way.  

Solution:

This was not easy to find, But what you will need to is set (Examples below):

  • "index_patterns": ["*"]  /* this will match all indexes created */
  • "mappings"."{type}".  This should be set to "_default_"

Note: this does not effect existing Templates.

PUT _template/template_1

{ 
    "index_patterns": ["*"],
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 1
    },
    "mappings": {
        "_default_": {
            "_source": {
                "enabled": true
            },
            "dynamic_templates": [
                {
                    "id_integers": {
                        "match": "*_id",
                        "match_mapping_type": "long",
                        "mapping": {
                            "type": "long"
                        }
                    }
                },
                {
                    "date_fields": {
                        "match": "*_date",
                        "match_mapping_type": "date",
                        "mapping": {
                            "type": "date"
                        }
                    }
                },
                {
                    "json_fields": {
                        "match": "*_json",
                        "match_mapping_type": "*",
                        "mapping": {
                            "type": "text"
                        }
                    }
                },
                {
                    "int_template": {
                        "match": "_*",
                        "match_mapping_type": "string",
                        "mapping": {
                            "type": "keyword"
                        }
                    }
                }                
            ],
            "properties": {
                "creation_date": {
                    "type": "date",
                    "format": "yyyy-MM-dd"
                },              
                "timestamp": {
                    "type": "date",
                    "format": "epoch_millis"
                }
            }
        }
    }
}

Test:  POST /school/student/1

{ 
  "title": "The quick brown fox", 
  "price": 5, 
  "colors": ["red","green","blue"],
  "do_something":"hello world"
}

Validation:  POST /school/student/_mapping

{
  "school": {
    "mappings": {
      "student": {
        "dynamic_templates": [
          {
            "id_integers": {
              "match": "*_id",
              "match_mapping_type": "long",
              "mapping": {
                "type": "long"
              }
            }
          },
          {
            "date_fields": {
              "match": "*_date",
              "match_mapping_type": "date",
              "mapping": {
                "type": "date"
              }
            }
          },
          {
            "json_fields": {
              "match": "*_json",
              "mapping": {
                "type": "text"
              }
            }
          },
          {
            "int_template": {
              "match": "_*",
              "match_mapping_type": "string",
              "mapping": {
                "type": "keyword"
              }
            }
          }
        ],
        "properties": {
          "creation_date": {
            "type": "date",
            "format": "yyyy-MM-dd"
          },
          "timestamp": {
            "type": "date",
            "format": "epoch_millis"
          }
        }
      }
    }
  }
}

Mac Sierra - Amazon AWS CLI Easy Installer

http://docs.aws.amazon.com/cli/latest/userguide/awscli-install-bundle.html

    Switch to the download folder

    $ cd ~/Downloads

    Download the AWS CLI Bundled Installer.

    $ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"

    Unzip the package.

    $ unzip awscli-bundle.zip

    Note

    If you don't have unzip, use your Linux distribution's built in package manager to install it.

    Run the install executable.

    $ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws

    Note

    By default, the install script runs under the system default version of Python. If you have installed an alternative version of Python and want to use that to install the AWS CLI, run the install script with that version by absolute path to the Python executable. For example:

    $ sudo /usr/local/bin/python2.7 awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws