Splunk : queries

1.SPL Keywords 

By | As | over | where|  or |

2.Command chaining

index ="AirBNB" | where Price > 40

index="AirBnB" Neighborhood Queens "Property Type"= Apartment | where Price >70 or Price =65
________________________________________________________________
3.SPL Filtering and Modifying 

• Field
• Search
• Rename

Fields : Add/Subtract fields that we want to show in our search

index="AirBnB Neighborhood =Queens "Property Type"= Apartment | field -bed

index="AirBnB Neighborhood =Queens "Property Type"= Apartment | field +bed,price

search : searches for raw text

index="AirBnB Neighborhood =Queens "Property Type"= Apartment | search large

rename : rename a field for raw text

index="AirBnB Neighborhood =Queens "Property Type"= Apartment | search large |rename Price as Cost

________________________________________________________________

Ordering

1. Head
2. Tail
3. sort
4. reverse
5. Table 

Head

index="AirBnB Neighborhood =Queens "Property Type"= Apartment | search large |head 20

Tail

index="AirBnB Neighborhood =Queens "Property Type"= Apartment | search large |tail 30

sort

index="AirBnB Neighborhood =Queens "Property Type"= Apartment | search large |sort Price desc

reverse

index="AirBnB Neighborhood =Queens "Property Type"= Apartment | search large |rename Price as Cost

Table 

index="AirBnB Neighborhood =Queens "Property Type"= Apartment | search large |rename Price as Cost

_______________________________________________________________

Transformative commands allow search commands to create data structure from field values.

Top : most frequent values
index=airbnb | top limit 3 beds
Rare: least frequent values
index=airbnb | rare limit 3 beds
Highlight
Contingency
index=airbnb | contingency Neighborhood  "Property Type"

Stats commands

Special type of transforming commands used primarily for calculations to for table results

|stats avg([fieldname])
calculate results based on average function
index=airbnb | stats avg(Price)
index=airbnb | stats avg(Price) by Neigbourhood 

Find number of listings by neighborhood
|stats count(eval[fieldname]) as new name . : returns timeseries chart over the data element 
index=airbnb | stats count

index=airbnb | stats count by Neigbourhood 

Find maximum  listing by price neighborhood
|stats max(fieldname) : returns maximum value for a specific field index=airbnb | stats max(Price)
index=airbnb | stats mav(Price) by Neighbourhood


Find min of listings by neighborhood
|stats min(fieldname) : returns minimum value for a specific field
index=airbnb | stats min(Price) 
index=airbnb | stats min(Price) by Neighbourhood

Find maximum of listings in neighborhood

|stats sum(fieldname) : returns sum value for a specific field

Chart commands

Type of splunk transforming command for presenting data in tables or visualization . Typically include stats commands

|chart somefunction([fieldname]) as new name  

|timechart somefunction([fieldname]) as new name 


Find the type of property listed by property type

index=airbnb | chart count as types by "Property Type"

host="tvidushi" | timechart count as messages  by keywords 

What is SPLUNK LOOK UP ?

Splunk is a software used to search and analyze machine data. It has built-in features to recognize the data types, field separators and optimize the search processes. It also provides data visualization on the search results.

Splunk can read this unstructured, semi-structured or rarely structured data. After reading the data, it allows to search, tag, create reports and dashboards on these data. With the advent of big data, Splunk is now able to ingest big data from various sources, which may or may not be machine data and run analytics on big data.

source type detection

Understanding JMX, Spring JMX

What is JMX ?

Provides an architecture to manage resources dynamically at runtime.

JMX is used mostly in enterprise applications to make system configurable or to get the state of application 

At any point of time .Because the JMX technology is dynamic ,you can use it to monitor and manage resources as they are created, installed and implemented .You can also use the JMX technology to monitor

And manage the JVM.

What can you do with  JMX ?

• Manage resourses
• Manage JVM

What are components of  MX ?

The JMX technology can be divided into three levels, as follows:

• Instrumentation
• JMX agent - Mbean Server
• Remote management

Instrumentation

To manage resources using the JMX technology, you must first instrument the resources in the Java programming language. You use Java objects known as MBeans to implement the access to the resources' instrumentation. MBeans must follow the design patterns and interfaces defined in the JMX specification. Doing so ensures that all MBeans provide managed resource instrumentation in a standardized way.

 In addition to standard MBeans, the JMX specification also defines a special type of MBean called an MXBean.

 An MXBean is an MBean that references only a pre-defined set of data types. Other types of MBean exist, but this trail will concentrate on standard MBeans and MXBeans.

Once a resource has been instrumented by MBeans, it can be managed through a JMX agent. MBeans do not require knowledge of the JMX agent with which they will operate.

MBeans are designed to be flexible, simple, and easy to implement. Developers of applications, systems, and networks can make their products manageable in a standard way without having to understand or invest in complex management systems. Existing resources can be made manageable with minimum effort.

In addition, the instrumentation level of the JMX specification provides a notification mechanism. This mechanism enables MBeans to generate and propagate notification events to components of the other levels.

JMX Agent

•  is a standard management agent that directly controls resources and makes them available to remote management applications. JMX agents are usually located on the same machine as the resources they control, but this arrangement is not a requirement.
• The core component of a JMX agent is the MBean server, a managed object server in which MBeans are registered.
•  A JMX agent also includes a set of services to manage MBeans, and 
• at least one communications adaptor or connector to allow access by a management application.

When you implement a JMX agent, you do not need to know the semantics or functions of the resources that it will manage. In fact, a JMX agent does not even need to know which resources it will serve because any resource instrumented in compliance with the JMX specification can use any JMX agent that offers the services that the resource requires. Similarly, the JMX agent does not need to know the functions of the management applications that will access it.

Remote Management

JMX technology instrumentation can be accessed in many different ways, either through existing management protocols such as the Simple Network Management Protocol (SNMP) or through proprietary protocols. The MBean server relies on protocol adaptors and connectors to make a JMX agent accessible from management applications outside the agent's Java Virtual Machine (Java VM).

Each adaptor provides a view through a specific protocol of all MBeans that are registered in the MBean server. For example, an HTML adaptor could display an MBean in a browser.

Connectors provide a manager-side interface that handles the communication between manager and JMX agent. Each connector provides the same remote management interface through a different protocol. When a remote management application uses this interface, it can connect to a JMX agent transparently through the network, regardless of the protocol. The JMX technology provides a standard solution for exporting JMX technology instrumentation to remote applications based on Java Remote Method Invocation (Java RMI).

Monitoring an Application by Using JConsole

This procedure shows how to monitor the Notepad Java application. Under releases of the Java SE platform prior to version 6, applications that you want to monitor with JConsole need to be started with the following option.

-Dcom.sun.management.jmxremote

However, the version of JConsole provided with the Java SE 6 platform can attach to any local application that supports the Attach API. In other words, any application that is started in the Java SE 6 HotSpot VM is detected automatically by JConsole, and does not need to be started using the above command-line option.

1. Start the Notepad Java application, by using the following command in a terminal window:

   java -jar 
       jdk_home/demo/jfc/Notepad/Notepad.jar
   

   Where jdk_home is the directory in which the Java Development Kit (JDK) is installed. If you are not running version 6 of the Java SE platform, you will need to use the following command:
   

   java -Dcom.sun.management.jmxremote -jar 
         jdk_home/demo/jfc/Notepad/Notepad.jar
   

2. Once Notepad has opened, in a different terminal window, start JConsole by using the following command:

   jconsole
   

   A New Connection dialog box is displayed.
3. In the New Connection dialog box, select Notepad.jar from the Local Process list, and click the Connect button.JConsole opens and connects itself to the Notepad.jar process. When JConsole opens, you are presented with an overview of monitoring and management information related to Notepad. For example, you can view the amount of heap memory the application is consuming, the number of threads the application is currently running, and how much central procesing unit (CPU) capacity the application is consuming.
4. Click the different JConsole tabs.Each tab presents more detailed information about the different areas of functionality of the Java VM in which Notepad is running. All the information presented is obtained from the various JMX technology MXBeans mentioned in this trail. All the platform MXBeans can be displayed in the MBeans tab. The MBeans tab is examined in the next section of this trail.
5. To close JConsole, select Connection -> Exit.

MBEAN

This lesson introduces the fundamental concept of the JMX API, namely managed beans, or MBeans.

An MBean is a managed Java object, similar to a JavaBeans component, that follows the design patterns set forth in the JMX specification. An MBean can represent a device, an application, or any resource that needs to be managed. MBeans expose a management interface that consists of the following:

• A set of readable or writable attributes, or both.
• A set of invokable operations.
• A self-description.

The management interface does not change throughout the life of an MBean instance. MBeans can also emit notifications when certain predefined events occur.

The JMX specification defines five types of MBean:

• Standard MBeans
• Dynamic MBeans
• Open MBeans
• Model MBeans
• MXBeans

The examples in this trail demonstrate only the simplest types of MBean, namely standard MBeans and MXBeans.

ESRI-latest

What is ArcGIS Online?

ArcGIS Online is a cloud-based mapping and analysis solution. Use it to 

1. Make maps, 
2. Analyze data, and to share and collaborate. 
3. Get access to workflow-specific apps, maps and data from around the globe, and tools for being mobile in the field. Your data and maps are stored in a secure and private infrastructure and can be configured to meet your mapping a requirements.
4. https://doc.arcgis.com/en/arcgis-online/reference/what-is-agol.htm

1.Get started with Maps?

1. Choose base maps maps, 
2. Add layers
3. Change Styles
4. Configure popups 
5. Save Map

2.Get started with Scenes?

1. Choose global or local scene
2. Switch basemap
3. Add layers 
4. Configure Content
5. Capture slides
6. Save scene

https://doc.arcgis.com/en/arcgis-online/get-started/get-started-with-scenes.htm

http://bhuvan.nrsc.gov.in/gis/thematic/index.php

3.Get started with apps

1. Pick type of  app
2. Build Map ,Scenes,Groups
3. Assemble your content
4. Refine 
5. Choose and publish
6. Save scene

https://doc.arcgis.com/en/arcgis-online/get-started/get-started-with-scenes.htm

4.Get started with analysis

1. Ask and explore
2. Model and Compute
3. Examine and Interpret
4. Make decisions
5. Share decisions

5.Get started with sharing

https://doc.arcgis.com/en/arcgis-online/get-started/get-started-with-sharing.htm

6.Get started with groups

https://doc.arcgis.com/en/arcgis-online/get-started/get-started-with-groups.htm

7.ArcGIS REST Service Directory

https://services.arcgis.com/V6ZHFr6zdgNZuVG0/ArcGIS/rest/services

What is BaseMap?

A basemap provides a background of geographical context for the content you want to display in a map. When you create a new map, you can choose which basemap you want to use. You can change the basemap of the current map at any time using the basemap gallery or your own layer as the basemap. You can also create a basemap containing multiple layers from the Contents pane in Map Viewer.

 https://doc.arcgis.com/en/arcgis-online/create-maps/choose-basemap.htm

_________________________________________________________________________________
ESRI

BASEMAP

Creates a new basemap object. Basemaps can be created from a PortalItem, from a well known basemap ID, or can be used for creating custom basemaps. These basemaps may be created from tiled services you publish to your own server, or from tiled services published by third parties.

CAMERA

The camera defines the position, tilt, and heading of the point from which the SceneView's visible extent is observed. It is not associated with device hardware. This class only applies to 3D SceneViews.

When a SceneView is created, it contains a camera property. Since the camera property may be autocast, this module does not need to be included in the require() function in most applications.

COLOR

Creates a new color object by passing either a hex, rgba, or named color value. This class inherits all attributes from dojo/_base/Color to provide functions for setting colors.

GEOMETRY

A convenience module for importing Geometry classes when developing with TypeScript. For example, rather than importing geometries one at a time like this:

GRAPHIC

A Graphic is a vector representation of real world geographic phenomena. It can contain geometry, a symbol, and attributes. A Graphic is displayed in the GraphicsLayer.

GROUND

The Ground class contains properties that specify how the ground surface is displayed in a SceneView. It contains a layers property, which is a collection of ElevationLayer that defines the elevation or terrain of the map's surface.

MAP

The Map class contains properties and methods for storing, managing, and overlaying layers common to both 2D and 3D viewing. Layers can be added and removed from the map, but are rendered via a MapView (for viewing data in 2D) or a SceneView (for viewing data in 3D). Thus a map instance is a simple container that holds the layers, while the View is the means of displaying and interacting with a map's layers and basemap.

A single map may be referenced by multiple views. This sample for example, contains a single Map that is visible in two separate views - one in 2D and the other in 3D. Because one map may be accessed by multiple views in the same application, all user interaction with a map's layers is handled on the View, not the Map.

An instance of Map is an essential component of the MapView and SceneView. A Map object should be created prior to a view so it can be passed into the map property of that view (e.g. MapView.map, SceneView.map).

pointCloudRenderers

A convenience module for importing PointCloudRenderer classes when developing with TypeScript. For example, rather than importing renderers one at a time like this:

BaseDynamicLayer

______________________________

esri/geometry/

_____________________________________________________________________

Circle

A circle is a Polygon created by specifying a center point and a radius. The point can be provided as a Point object or an array of latitude/longitude values.

_________________________________________________________________________

coordinateFormatter

Extent

The minimum and maximum X and Y coordinates of a bounding box. Extent is used to describe the visible portion of a MapView. When working in a SceneView, Camera is used to define the visible part of the map within the view.

Geometry

The base class for geometry objects. This class has no constructor. To construct geometries see Point, Polyline, or Polygon.

geometryEngine

A client-side geometry engine for testing, measuring, and analyzing the spatial relationship between two or more 2D geometries. If more than one geometry is required for any of the methods below, all geometries must have the same spatial reference for the methods to work as expected. Read the following blog series to learn more about GeometryEngine:

geometryEngineAsync

An asynchronous client-side geometry engine for testing, measuring, and analyzing the spatial relationship between two or more 2D geometries. If more than one geometry is required for any of the methods below, all geometries must have the same spatial reference for the methods to work as expected. Read the following blog series to learn more about GeometryEngine:

______________________________________________________________________________
ESRI RELATED LINKS

https://stackoverflow.com/questions/22777734/what-is-initial-scale-user-scalable-minimum-scale-maximum-scale-attribute-in

https://developers.arcgis.com/javascript/latest/guide/get-api/index.html



ESRI published Services


DRAG AND DROP DATA TO SERVICE

https://developers.arcgis.com/javascript/3/samples/exp_dragdrop/

__________________________________________________________________________

Approaches for publishing services with Arc GIS

http://enterprise.arcgis.com/en/server/latest/publish-services/windows/approaches-for-publishing-services-with-arcgis.htm

__________________________________________________________________________



__________________________________________________________________________

__________________________________________________________________________


__________________________________________________________________________


__________________________________________________________________________


__________________________________________________________________________

__________________________________________________________________________
INTRODUCTION

https://developers.arcgis.com/javascript/latest/sample-code/intro-layers/index.html





__________________________________________________________________________
INTRODUCTION 

https://developers.arcgis.com/javascript/latest/sample-code/intro-layers/index.html



__________________________________________________________________________

https://developers.arcgis.com/javascript/latest/sample-code/sandbox/index.html?sample=intro-layers

__________________________________________________________________________

https://developers.arcgis.com/javascript/latest/sample-code/sandbox/index.html?sample=layers-custom-dynamiclayer

__________________________________________________________________________




https://developers.arcgis.com/javascript/latest/sample-code/sandbox/index.html?sample=layers-custom-dynamiclayer




_________________________________________________________________________________



ESRI BLOGS

https://www.esri.com/arcgis-blog/products/arcgis-online/mapping/map-multiple-attributes-at-once-using-predominance/

SPRING SECURITY

    <parent>

        <groupId>org.springframework.boot</groupId>

        <artifactId>spring-boot-starter-parent</artifactId>

        <version>1.4.3.RELEASE</version>

    </parent>

      <!-- Add typical dependencies for a web application -->

        <!-- Adds Tomcat and Spring MVC, along others, jackson-databind included transitively -->

        <dependency>

            <groupId>org.springframework.boot</groupId>

            <artifactId>spring-boot-starter-web</artifactId>

        </dependency>

import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.http.HttpHeaders;

import org.springframework.http.HttpStatus;

import org.springframework.http.ResponseEntity;

import org.springframework.web.bind.annotation.PathVariable;

import org.springframework.web.bind.annotation.RequestBody;

import org.springframework.web.bind.annotation.RequestMapping;

import org.springframework.web.bind.annotation.RequestMethod;

import org.springframework.web.bind.annotation.RestController;

Spring Boot OAuth2 Part 1 - Getting The Authorization Code

https://www.javainuse.com/spring/spring-security-interview-questions
https://www.javainuse.com/spring/spring-boot-oauth-introduction
https://www.javainuse.com/spring/spring-boot-oauth-authorization-code
https://www.javainuse.com/spring/spring-boot-oauth-access-token


Spring security FLOW:

https://dzone.com/articles/spring-security-authentication

https://www.dineshonjava.com/spring-security-java-based-configuration-with-example/

Multiple Authentication Providers in Spring Security

2. The Authentication Provider

https://www.logicbig.com/tutorials/spring-framework/spring-security/custom-authentication-provider.html

https://www.baeldung.com/spring-security-authentication-provider

Spring 4 Security Features

Spring 3.x Security Framework provides the following Features:

1. Authentication and Authorization.
2. Supports BASIC,Digest and Form-Based Authentication.
3. Supports LDAP Authentication.
4. Supports OpenID Authentication.
5. Supports SSO (Single Sign-On) Implementation.
6. Supports Cross-Site Request Forgery (CSRF) Implementation.
7. Supports “Remember-Me” Feature through HTTP Cookies.
8. Supports Implementation of ACLs
9. Supports “Channel Security” that means automatically switching between HTTP and HTTPS.
10. Supports I18N (Internationalisation).
11. Supports JAAS (Java Authentication and Authorization Service).
12. Supports Flow Authorization using Spring WebFlow Framework.
13. Supports WS-Security using Spring Web Services.
14. Supports Both XML Configuration and Annotations. Very Less or minimal XML Configuration.

Spring 4.x Security Framework supports the following New Features:

1. Supports WebSocket Security.
2. Supports Spring Data Integration.
3. CSRF Token Argument Resolver.

We will develop some simple examples to demonstrate these features in my coming posts.

WHAT IS SPRING SECURiTY ?

1. MULTI FACTOR AUTHENTICATION
2. LOGIN 
3. AUTHERIZATION 
4. FILTERS
5. SECURITY SECRETS

 Spring security starts 

DelegatefilterProxy which 

1. Manages filters
2. is declared in web.xml

--> In Spring Boot, the most of the configurations will be auto configured and we do not have to worry about it. Therefore DelegatingFilterProxy will also be auto declared and configured. if you look at SecurityFilterAutoConfiguration class, you will find the following method which will register the filter named springSecurityFilterChain

Spring makes use of the DelegatingFilterProxy for implementing security mechanisms. It is a Proxy for standard Servlet Filter, delegating to a Spring-managed bean that implements the Filter interface. Its the starting point in the springSecurityFilterChain which instantiates the Spring Security filters according to the Spring configuration
Some of the features of Spring Security are

• Comprehensive and extensible support for both Authentication and Authorization
• Protection against attacks like session fixation, clickjacking, cross site request forgery, etc
• Servlet API integration Optional integration with Spring Web MVC

Security FilterChain interface  has two methods

•  matches() : check if the request applies to this filter chain

• getFilters() : returns all filters avaliable

SecurityContext Interface  has two methods

• getAuthentication()
• setAuthetication(Authetication authetication)

AutheticationManager  has a method

Authentication authenticate(Authentication authentication) throws AutheticationException

supports()

https://www.logicbig.com/tutorials/spring-framework/spring-security/custom-authentication-provider.html

Authetication Interface   extends prinicpal  has a method

• isAuthenticated()
• Object getPrinicpal()
• Object getCredentials()
• getAuthorities()

UserDetailsService()

loadUserByusername()

What's the difference between @Secured and @PreAuthorize in spring security 3?

https://stackoverflow.com/questions/3785706/whats-the-difference-between-secured-and-preauthorize-in-spring-security-3?rq=1

Spring Security Custom Authentication - AuthenticationProvider vs UserDetailsService

https://stackoverflow.com/questions/31630818/spring-security-custom-authentication-authenticationprovider-vs-userdetailsser?rq=1

Override
protected void configure(HttpSecurity http) throws Exception {
    http.csrf().disable()
            .authorizeRequests()
            .antMatchers("/").permitAll()
            .antMatchers("/register/*").permitAll()
            .antMatchers(HttpMethod.POST, "/login/manager").permitAll()
            .antMatchers("/hello").hasAuthority("AUTH_WRITE")
            .antMatchers("/world").hasRole("ADMIN")
            .anyRequest().authenticated()
            .and()
            .addFilterBefore(new JWTLoginFilter("/manager/login", authenticationManager()),
                    UsernamePasswordAuthenticationFilter.class)
            .addFilterBefore(new JWTAuthenticationFilter(),
                    UsernamePasswordAuthenticationFilter.class);

}

How to get the AuthenticationManager when using the AuthenticationManagerBuilder to add custom provider?

https://stackoverflow.com/questions/49410977/how-to-get-the-authenticationmanager-when-using-the-authenticationmanagerbuilder?rq=1


ssssssss

TYPES OF SECURITY

Declarative Security - Declarative security specifies an application's security requirements by using either deployment descriptors or annotations.

e.g

Programmatic Security - Programmatic security implements an application's security within the application code.

KEY CHARACTERSTICS 

1.Following are the key characteristics of application security.

Authentication - Authentication is the means by which a user or client proves to a server that it is authorized to access a specific resource and vice-versa.

Authorization - Authorization is the means by which a server determines if a user has permissions to access a specific resource or data.

Data Integrity - Data integrity means that the data that is exchanged by a client and server is not modified by an unauthorized third party.

Confidentiality or Data privacy - This ensures that information is send to only those users or clients that are authorized to access the data.

Non-repudiation - This means that you can prove that a transaction or action has occurred. So a user who has performed a certain action, cannot deny doing so.

SECURITY MECHANISM

Application-Layer Security - In Java EE applications, the application-layer security is provided by the component containers.

Transport-Layer Security - Transport-Layer security is provided by the transport mechanism used to transmit data between the client and server. Java EE application relies on the secure HTTPS protocol using Secure Sockets Layer (SSL).

Message-Layer Security - Message-Layer security secures the SOAP messages that are exchanged between client and server using XML web services 

Realms, Users, Groups and Roles

Realms [a field or domain of activity or interest.]- Realms are security domains or protection spaces setup for web or application servers. Each realm has its own authentication scheme and contains a collection of Users and Groups.

Users - Users are individual or application entities defined in an identity store that access the application resources.

Group - Groups are abstract entities defined in Java EE that contains a set of users having common traits.

           Roles - Roles are are abstract entities defined in Java EE that has permissions to access a set of    secured resources in an application. Users or Groups are mapped to Roles. 

AUTHENTICATION MECHANISM

1.  BASIC AUTHTICATION

2.  FORM BASED AUTHETICATION

3.  DIGEST AUTHTITCATION

4.CLIENT-CERT

5. KERBEROS

6.OUTH 2