splunk query neighboring events

splunk query neighboring events

  1. you should be able to -A or -B (but not both) using the transaction command
  2. equivalent of -B .... | transaction endswith=(<search that matches the event of interest>) maxevents=<number of events in txn>
  3. equivalent of -A
  4. .... | transaction startswith=(<search that matches the event of interest>) maxevents=<number of events in txn>

Continue reading

Advertisements
Posted in Uncategorized | Leave a comment

Unique Binary search Tree

Problem

Given an integer n, generate all structurally unique BST’s (binary search trees) that store values 1…n.

For example,
Given n = 3, your program should return all 5 unique BST’s shown below.

1         3     3      2      1
\       /     /      / \      \
 3     2     1      1   3      2
/     /       \                 \
2     1         2                 3

Solution 1

Continue reading

Posted in Uncategorized | Leave a comment

Manacher’s algorithm

manacher’s algorithm

what’s it?

manacher’s algorithm is good at calculate palindrome string, the time-complexity can reach o(n). it would re-use the pattern to calcuate, so it is very efficient.

case 1 : sub-palindrome is contained in parent palindrome
case 1

case 2: sub-palindrome is over parent palindrome
case 2

Code

package com.weixin;

public class ManachersAlgorithm {

    public  int findLongestPalindrome(String s) {
        if(s==null || s.length() <= 1){
            return s.length();
        }

        int[] p = new int[s.length()];
        p[0] = 1;
        int idx = 0, rMax = 0;
//        S  #  1  #  2  #  2  #  1  #  2  #  3  #  2  #  1  #
//        P  1  2  1  2  5  2  1  4  1  2  1  6  1  2  1  2  1
        for(int i = 1; i < s.length(); i++){
             /** --left------j------idx------i--------rMax-----
                *idx means the center which has the maximum radius of palindrome , include himself
                *p[i] means the radius of the palindrome center on i, include i itself
                *so rMax = idx + p[idx] - 1 ( since include idx itself, so we need to minus 1)
                *for example when i is 7, then p[7] = 4, rMax = 4+7 - 1 = 10,  1 # 2 #(rMax is here)
                *i is the current index, j is the index which is symetric pointer centered on idx
                *since i - idx = idx -j , then we get j = 2 * idx - i, but the pre-requsite is 2*idx -i >= 0
                *if rMax - i > p[j], what does that mean? 
                *firstly, we already know p[j], secondly we know p[idx] and rMax, let's give an example
                *|------|----j---|----^-----i------|------|
                *|------l----j===p----idx-------i===p'---r------|
                *assume p[j] is at the position p, i is symetric, assume p[i] is at position p', since i and j are symetric,
                *we know dist(j,p) = p[j] == dist(i,p'), if p' is between i and rMax,
                *which means p[j] is totally contained in the long palindrome, and we can directly know
                *that p[i] = p[j], then we don't need to calculate any more.
                *for ex,
                * 
                * S  #  1  #  2  #  2  #  1  #  2  #  3  #  2  #  1  #
                * P  1  2  1  2  5  2  1  4  1  2  1  6  1  2  1  2  1
                * ------------j--^--i--------r----
                * we know p[j] is 2, p[idx]=5, rMax = idx+p[idx]-1 = 4+5-1 = 8, j = 3, idx=4, i = 5
                * rMax - i = 8-5 = 3 > p[j] = 2, so we directly know p[5] = p[3] = 2
                * 
                *another case is j + p[j] > idx, what it looks like : 
                *|------|----j------^-----|-----i------|-----|
                *|------l----j------idx---p-----i------r-----|
                *in this case, rMax -i < p[2*idx-i], then we can not directly get p[i], but we still can re-use p[j]
                *we already know centered on i, all the way to rMax it is palindrome, so we just need to calculate start from rMax
                *this also save some calculation

                *
             **/

            if( 2 * idx >= i && rMax - i > p[2*idx - i]){
                p[i] = p[2*idx-i];
            }
            else{
               /**
                * if i is already reach or over the rMax boader, we need to re-calcuate
                * at below, we need to re-use p[rMax-i], so we need to gurantee rMax-i >= 0
                */
               if(i >= rMax){
                   idx = i;
                   p[i] = getPalLen(s, i);
                   rMax = i + p[i]-1;
               }
               else{
                   /**
                    * as we explained above, if p[j] is so big, which make the rMax - i < p[j(j=2*idx-i)]
                    * in this case, at least we know start from i to rMax, it must be a palindrome, and we can re-use this result
                    * so we have p[i] = p[rMax -i]
                    */
                   p[i] = p[rMax -i];
                   /**start from i+p[i], we check palindrome,go left and right, compare one by one, then we increase p[i] accordingly
                    *  but we need to be aware that i+p[i] < s.length, I made a mistakes here, debug it for quite some time.
                    *  
                    */ 
                   while(i+p[i] < s.length() && s.charAt(i+p[i])==s.charAt(i-p[i])){
                       p[i]++;

                   }
                   /**
                    * if i+p[i] larger than rMax already, then update rMax
                    */
                   if(i+p[i] > rMax){
                       idx = i;
                       rMax = i + p[i]-1;
                   }
               }
            }

        }

        int k = 0;
        for(int a : p){
            k = Math.max(k,a);
        }
        return k -1;
    }

    public int getPalLen(String s, int i){
        int l = i, r = i;
        while(l>=0 && r <= s.length()-1){
            if(s.charAt(l)!=s.charAt(r)){
                break;
            }
            l--;
            r++;
        }
        return r-i;
    }

    public String specialHandler(String s){
        StringBuilder sb  = new StringBuilder();
        sb.append('#');
        for(char c : s.toCharArray()){
            sb.append(c);
            sb.append('#');
        }
        return sb.toString();
    }

    public static void main(String[] args) {
        // TODO Auto-generated method stub
        ManachersAlgorithm ma = new ManachersAlgorithm();
        String s = "12212321";
        String s1 = ma.specialHandler(s);
        int k = ma.findLongestPalindrome(s1);
        System.out.println(k);
        s = "abaxabaxabb";
        s1 = ma.specialHandler(s);
        k = ma.findLongestPalindrome(s1);
        System.out.println(k);
        s = "abaxabaxabybaxabyb";
        s1 = ma.specialHandler(s);
        k = ma.findLongestPalindrome(s1);
        System.out.println(k);

    }

}
Posted in algorithm | Leave a comment

a speical character issue on jenkins jobdsl plugin

a speical character issue on jenkins jobdsl plugin

Issue

I change one jobDsl groovy file for jenkins, try to add some shell script there :

&lt;br /&gt; shell (
&quot;&quot;&quot;
#!/bin/bash
pid=$(netstat -tulnp | awk -vport=28189 '$4~&quot;:&quot;port&quot;{print 0+$7}')
echo &quot;$pid&quot;
if [[ -n $pid ]]; then
  kill -9 &quot;$pid&quot;
fi
cd app/sdk
/opt/maven/bin/mvn clean deploy
“”&quot;

but when I tried to regnerate the jenkins job, I got this error :

&lt;br /&gt;ERROR: Build step failed with exception
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1.groovy: 13: illegal string body character after dollar sign;
   solution: either escape a literal dollar sign &quot;\$5&quot; or bracket the value expression &quot;${5}&quot; @ line 13, column 11.
       shell (
             ^

1 error

    at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:302)
    at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:149)
    at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:119)
    at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:131)
    at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:359)
    at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:137)
    at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:108)
    at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:236)
    at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:161)
    at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:846)
    at org.codehaus

the problem is here : `

either escape a literal dollar sign “\$5” or bracket the value expression “${5}” @ line 13, column 11.
`

Solution

the problem caused by the $ sign, to fix it, here is the solution:

&lt;br /&gt;#!/bin/bash
pid=\$(netstat -tulnp | awk -vport=28189 '\$4~&quot;:&quot;port&quot;{print 0+\$7}')
echo &quot;\$pid&quot;
if [[ -n \$pid ]]; then
  kill -9 &quot;\$pid&quot;
fi

see this link dollar slashy ring

test

&lt;br /&gt;$ ./gradlew clean jobDsl
$ ls
README.md             gradle                gradlew               run.jenkins.docker.sh
build.gradle          gradle.properties     gradlew.bat           src
Then you can find your generated job xml here:
$ less build/jobs/qbo-test-master-build-commit.xml
Posted in devops | Tagged | Leave a comment

Setup p4merge as a visual diff and merge tool for git Raw

Download and install p4merge

Setup p4merge as a visual mergetool

$ git config --global merge.tool p4mergetool
$ git config --global mergetool.p4mergetool.cmd \
"/Applications/p4merge.app/Contents/Resources/launchp4merge \$PWD/\$BASE \$PWD/\$REMOTE \$PWD/\$LOCAL \$PWD/\$MERGED"
$ git config --global mergetool.p4mergetool.trustExitCode false
$ git config --global mergetool.keepBackup false

Setup p4merge as a visual diff tool

$ git config --global diff.tool p4mergetool
$ git config --global difftool.p4mergetool.cmd \
"/Applications/p4merge.app/Contents/Resources/launchp4merge \$LOCAL \$REMOTE"

Using p4merge to resolve conflicts

When you run into a conflict when merging simply run:

$ git mergetool
  • You will be prompted to run “p4mergetool”, hit enter and the visual merge editor will launch.
  • Using the merge tool you can resolve the conflict and then save the file.
  • After you exit the merge tool take a look back at your terminal. You will be asked if the merge was successful, choose yes if all is well or no if you need to start over.
  • This prompting is happening because the “trustExitCode” option is turned off. Personally I always like to be asked, but you can have git just trust the exit code from the merge tool.
Posted in work related | Tagged | Leave a comment

what to think when you write a binary search

Binary search

binary search is prety tricky and have a lot hidden dents, here is my summary.

Common problem

given a non-descending array, find element with value m in the array.

below is the classic c code

int binary_search(int *A, int n, int target)
{
    int low = 0, high = n - 1;
    assert(A != NULL && n >= 0);
    while (low <= high)
    {
        int mid = low + (high - low) / 2;
        if(A[mid] == target)
            return mid;
        else if (A[mid] < target)
            low = mid + 1;
        else
            high = mid - 1;
    }
    return -1;
}

there are several places you need to consider, which is shown below : bs1
there are totally 6 places you need to be aware whey you write to binary search.

  • for 1 and 2, I choose the close bounary 0 and n-1.
  • for 3, I choose `low a[mid]= 7, so low = 3.
    1. low = 3, mid = 4, high = 5, 10 > a[mid] = 9, so low = 10
    2. if the while loop check is low &gt;1, but there are some risks there. if low and high is big int, then the sum would possibly overpass the INT_MAX. so it is safe to use the format at place 4. but why not use int mid = high - (high-low) &gt;&gt; 1? In fact, in some case, we have to use it that way.
  • for place 5 and place 6, they have differnet form, you might use low = mid or low = mid+1, for high, it might be high = mid -1 or hign = mid, it is case by case.

basically, all the binary search problem is somehow related to how to find the proper combination for the 6 places. So Here I will give example.

Problem 1

given a sorted(non-ascending) array A, it may contains repeated elements, get the smallest i which make A[i]=target, if not, return -1.

int searchFirstPos(int A[], int n, int target)
{
  if(n <= 0) return -1;
  int low = 0, high = n-1;
  while(low < high) // 1 - is low <= high doable?
  {
    int mid = low+((high-low)>>1);
    if(A[mid] < target)
      low = mid+1;
    else // A[mid] >= target
      high = mid; // 2, why not high = mid -1 ?
  }
  /* 
  循环过程中,当low大于0时,A[low-1]是小于target的,因为A[mid] < target时,
  low=mid+1;当high小于n-1时,A[high]是大于等于target的,因为A[mid] >= target时,
  high = mid;循环结束时,low 等于 high,所以,如果A[low](A[high])等于target,
  那么low(high)就是target出现的最小位置,否则target在数组中不存在。
  */
  if(A[low] != target)
    return -1;
  else
    return low;
}

question 1 : at the comment 1, can I use lowmid=0, soA[mid]==targt`, infinite loop.

see the leet code issue – Search Insert Position

Given a sorted array and a target value, return the index if the target is found. If not, return the index where it would be if it were inserted in order.

You may assume no duplicates in the array.

Here are few examples.

[1,3,5,6], 5 → 2
[1,3,5,6], 2 → 1
[1,3,5,6], 7 → 4
[1,3,5,6], 0 → 0

code :
“`c++
class Solution {
public:
int searchInsert(vector<int>& nums, int target) {

     if(nums.empty())
    {
        return 0;
    }
    int left = 0;
    int right = nums.size(); // or nums.size()-1, both are fine
    while(left&lt;right)
    {
        int mid = left + floor((right-left)/2);
        if(target &gt; nums[mid])
        {
            left = mid+1;
        }
        else
        {
            right = mid;
        }
    }
    if(left==nums.size()-1&amp;&amp;nums[left]&lt;target)
    {
        return left + 1;
    }
    return left;

}

};

#### Problem 2
> given a sorted(non-descending) array, may contains repeated elements, find the biggest i which makes `A[i] = target`, if not , return -1.

```c
int searchLastPos(int A[], int n, int target)
{ 
  if(n <= 0) return -1;
  int low = 0, high = n-1;
  while(low < high)
  {
    /*
    这里中间位置的计算就不能用low+((high-low)>>1)了,因为当low+1等于high
    且A[low] <= target时,会死循环;所以这里要使用low+((high-low+1)>>1),
    这样能够保证循环会正常结束。
    */
    int mid = low+((high-low+1)>>1);
    if(A[mid] > target)
      high = mid-1;
    else // A[mid] <= target
      low = mid;
  }
  /* 
  循环过程中,当high小于n-1时,A[high+1]是大于target的,因为A[mid] > target时,
  high=mid-1;当low大于0时,A[low]是小于等于target的,因为A[mid] <= target时,
  low = mid;循环结束时,low 等于 high,所以,如果A[high](A[low])等于target,
  那么high(low)就是target出现的最大位置,否则target在数组中不存在。
  */
  if(A[high] != target)
    return -1;
  else
    return high;
}

question 1 : why we cannot use int mid = low+((high-low)&gt;&gt;1);
answer : suppose you have test case A = [1,2], target = 2
– low = 0, high = 1, mid = 0
– A[mid] = 1 >1);, I would rather use int mid = high – ((high – low)>>1)`, which looks better.

so the code becomes like this :

int searchLastPos(int A[], int n, int target)
{ 
  if(n <= 0) return -1;
  int low = 0, high = n-1;
  while(low < high)
  {
    /*
    这里中间位置的计算就不能用low+((high-low)>>1)了,因为当low+1等于high
    且A[low] <= target时,会死循环;所以这里要使用low+((high-low+1)>>1),
    这样能够保证循环会正常结束。
    */
    int mid = high -  ((high-low)>>1);
    if(A[mid] > target)
      high = mid-1;
    else // A[mid] <= target
      low = mid;
  }
  /* 
  循环过程中,当high小于n-1时,A[high+1]是大于target的,因为A[mid] > target时,
  high=mid-1;当low大于0时,A[low]是小于等于target的,因为A[mid] <= target时,
  low = mid;循环结束时,low 等于 high,所以,如果A[high](A[low])等于target,
  那么high(low)就是target出现的最大位置,否则target在数组中不存在。
  */
  if(A[high] != target)
    return -1;
  else
    return high;
}

In fact, problme 2 is similar to the get_upperbound

Problem 3

given a sorted array, may contains duplicates, find a biggest i which makes A[i] < target, return i, if not, return -1.

problem 3 is ask for `A[i] target, return i, if not, return -1.

int searchFirstPosGreaterThan(int A[], int n, int target)
{
  if(n <= 0) return -1;
  int low = 0, high = n-1;
  while(low < high)
  {
    int mid = low+((high-low)>>1); // 1
    if(A[mid] > target)  // 2
      high = mid; // 3
    else // A[mid] <= target // 4
      low = mid+1; // 5
  }
  /* 
  循环过程中,当low大于0时,A[low-1]是小于等于target的,因为A[mid] <= target时,
  low=mid+1;当high小于n-1时,A[high]是大于target的,因为A[mid] > target时,
  high = mid;循环结束时,low 等于 high,所以,如果A[high](A[low])大于target,
  那么high(low)就是要找的位置,否则不存在这样的位置(A[n-1] <= target时)。
  */
  return A[high] > target ? high : -1;
}

Problem 5

given a sorted array (non-descending), may contains duplicates, try to find out how many times target appears at the array.

int count(int A[], int n, int target)
{
  int firstPos = searchFirstPos(A, n, target); // 第一次出现位置
  if(firstPos == -1)
    return 0;
  int lastPos = searchLastPos(A, n, target);  // 最后一次出现位置
  return lastPos-firstPos+1;  // 出现次数
}

there is also a similar problme in leet code : Search for a Range

Given a sorted array of integers, find the starting and ending position of a given target value.

Your algorithm’s runtime complexity must be in the order of O(log n).

If the target is not found in the array, return [-1, -1].

For example,
Given [5, 7, 7, 8, 8, 10] and target value 8,
return [3, 4].

“`c++
class Solution {
public:
vector<int> searchRange(vector<int>& nums, int target) {
vector<int> res;
if(nums.empty()) return res;
res.push_back(get_lowerbound(nums, target));
res.push_back(get_upperbound(nums,target));
return res;
}

<pre><code>int get_lowerbound(vector&lt;int&gt; &amp; nums, int target){
int left = 0, right = nums.size()-1;
while(left&lt;right){
int mid = left + (right-left)/2;
if(target &gt; nums[mid]){
left = mid+1;
}
else{
right = mid;
}
}
return (left &lt; nums.size() &amp;&amp; nums[right] == target) ? right : -1;
}

int get_upperbound(vector&lt;int&gt; &amp;nums, int target){
int left = 0, right = nums.size()-1;
while(left&lt;right){
int mid = right – (right-left)/2;
if(target &lt; nums[mid]){
right = mid-1;
}
else{
left = mid;
}
}

return (right &lt; 0 || nums[left] != target) ? -1 : left;
}
</code></pre>

};
“`

Posted in algorithm, Uncategorized | Tagged | Leave a comment

How to make nginx return 404 with json response

Problem

I need the the nginx to return a json response instead of the default static html page “nginx is not available temporarily.blabla..” if the content-tyhpe is “application/json”

in my case, nginx is used as a web app as well as an reverse proxy. its upstream server is a tomcat server. sometimes the tomcat server would down, then the user would get an 404. but most cases are rest api call, with a json content type. the client would expect return a json, an static html response would cause the client throw the json parser exception.

the archetecture of the aws project is like this:

How to

we need to use map directive in nginx.

  • I add a json file in my chef cookbook file/default folder, named 50x.json.
{
    "error": "temporarily_unailable",
    "error_description": "sorry, the site is under maintainance."
}
  • then in recipe/default.rb, I copy the file to var/www folder.
      cookbook_file "/var/www/50x.json" do
          source '50x.json'
          mode "0644"
      end
  • in the templates/default/nginx/vhost.d/default_application.conf.erb, I add those code snippet
server{
....

    error_page    500 502 503 504 $error_resp;
    location = /50x.html {
        root        /usr/share/nginx/html;
    }

    location = /50x.json {
        root        /var/www;
    }

}

map $http_content_type $error_resp {
    default /50x.html;
    ~json /50x.json;
}

be aware , the map directive must be in the http directive instead of server directive. I made this mistakes and stucked there for quite sometime.

see the results:
json :
json
html :
html

Posted in devops | Tagged , | Leave a comment