Showing posts with label request. Show all posts
Showing posts with label request. Show all posts

Friday, May 11, 2018

how to get a Network Panel show up when debugging Electron/Node applications

Leave a Comment

I am building an electron application and I am using libraries (request/axios) for making requests. One thing I didn't expect is that making these requests on Node won't display a Network Panel when running in chrome debug mode. Is there a simple way to tell debug mode to turn on a network panel for tuning into https requests(I assume these libraries all use https)?


currently on my server side electron app i only see the following enter image description here

1 Answers

Answers 1

Solution 1 - create your own

you can wrap your axios functions and send an event to your renderer process

main electron process

const electron = require('electron');  const {   app,   BrowserWindow,   ipcMain } = electron;  const _axios = require('request-promise');  const axios = {   get: (url, params) => _axios.get(url, params).then(sendData),   post: (url, params) => _axios.post(url, params).then(sendData),   delete: (url, params) => _axios.delete(url, params).then(sendData),   put: (url, params) => _axios.put(url, params).then(sendData)   // ... };  function sendData() {   return (data) => {     mainWindow.webContents.send('network', data);     return data;   }; } 

renderer process (index.html):

<!DOCTYPE html> <html>  <head>   <meta charset="UTF-8">   <title>Hello World!</title>    <link href="https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.1/css/bulma.min.css"          rel="stylesheet">   <style>     .kb-debug-widget {       position: fixed;       bottom: 0;       height: 200px;       overflow-x: hidden;       overflow-y: auto;       background: grey;       left: 0;       right: 0;       font-size: 10px;     }   </style> </head>  <body>   <div class="kb-debug-widget">     <table class="table is-bordered is-striped is-narrow is-hoverable is-fullwidth"            id="network">       <tr>         <th>Name</th>         <th>Method</th>         <th>Status</th>         <th>Type</th>         <th>Body</th>       </tr>     </table>   </div>   <script>     require('./renderer.js');     var {       ipcRenderer,       remote     } = require('electron');      ipcRenderer.on('network', (event, response) => {       const networkElement = document.getElementById('network');        // print whatever you want here!       networkElement.innerHTML +=         `       <tr>         <td>${response.request.href}</td>         <td>${response.request.method}</td>         <td>${response.statusCode}</td>         <td>${response.headers['content-type']}</td>         <td>${response. data}</td>       </tr>       `;        // you can also print the network requests to the console with a decent UI by using console.table:       console.table({         name: response.request.href,         method: response.request.method,         status: response.statusCode,         type: response.headers['content-type'],         body: response. data,       });     });   </script> </body>  </html> 

This will create a widget on the bottom of your view.

it's even easier with request:

const _request = require('request-promise'); const _axios = require('request-promise');  // this should cover all sub-methods const request = (params, callback) => {   return _request(params, callback)   .on('response', (response) => {     mainWindow.webContents.send('network', response);     return response;   }); }; 

Since both axios & request return similar objects, you can use the same function on the renderer side.

code in action

enter image description here

Here's a GitHub repository with the code implemented

Solution 1: Alt - write network requests to renderer's console

I also added an option to print the requests to the dev tools console, with console.table. Here's how it looks: enter image description here You can leave only this method if you don't want a widget inside your HTML.

Solution 2 - Run electron with the --inspect flag

You can also just run electron with the inspect flag, which allows you to debug your server code and has its own network tab with the "server-side" HTTP requests.

In order to see it, run your electron application like so:

electron --inspect=<port> your/app 

if you want to immediatly break on the first line, run the same command but replace --inspect with --inspect-brk.

After running the command, open any web-browser and go to chrome://inspect and selecting to inspect the launched Electron app present there. enter image description here

hope this helped. if you have any questions, you can ask me in the comments

Read More

Wednesday, September 27, 2017

Java: How to make API call with data?

Leave a Comment

I want to make API call similar to below curl command:

curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer  1djCb/mXV+KtryMxr6i1bXw"  -d '{"operands":[]}'  https://ads.line.me/api/v1.0/authority_delegations/get 

What I am trying

public void send_deligation_request(String details[]) throws Exception{     System.out.println(Arrays.toString(details));      URL line_api_url = new URL("https://ads.line.me/api/v1.0/authority_delegations/get");     String payload = "{operands:[]}";        HttpURLConnection linec = (HttpURLConnection)line_api_url.openConnection();     linec.setDoInput(true);     linec.setDoOutput(true);     linec.setRequestMethod("POST");     linec.setRequestProperty("Content-Type", "application/json");     linec.setRequestProperty("Authorization", "Bearer "+access_token);      OutputStreamWriter writer = new OutputStreamWriter(linec.getOutputStream(), "UTF-8");     writer.write(payload);       BufferedReader in = new BufferedReader(                             new InputStreamReader(                                     linec.getInputStream()));     String inputLine;      while ((inputLine = in.readLine()) != null)          System.out.println(inputLine);     in.close(); } 

But I am getting below error:

[naofumi.haida@torchlight.co.jp, 5514] 
java.io.IOException: Server returned HTTP response code: 400 for URL: https://ads.line.me/api/v1.0/authority_delegations/get   at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840)   at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)   at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)   at AuthorityDelegation.send_deligation_request(AuthorityDelegation.java:66)   at AuthorityDelegation.read_csv(AuthorityDelegation.java:36)   at AuthorityDelegation.main(AuthorityDelegation.java:20) 

Could somebody please help me?

5 Answers

Answers 1

HTTP code 400 means a BAD REQUEST.

I can't access the endpoint you have shared but here is free online REST API which I am using for demonstrating ..

curl -X POST \   https://jsonplaceholder.typicode.com/posts \   -H 'cache-control: no-cache' \   -H 'postman-token: 907bbf75-73f5-703f-c8b6-3e1cd674ebf7' \   -d '{         "userId": 100,         "id": 100,         "title": "main title",         "body": "main body"     }' 
  • -H = headers
  • -d = data

Sample Run:

[/c]$ curl -X POST \ >   https://jsonplaceholder.typicode.com/posts \ >   -H 'cache-control: no-cache' \ >   -H 'postman-token: 907bbf75-73f5-703f-c8b6-3e1cd674ebf7' \ >   -d '{ >         "userId": 100, >         "id": 100, >         "title": "main title", >         "body": "main body" >     }'    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                  Dload  Upload   Total   Spent    Left  Speed 100   258  100   150  100   108    147    106  0:00:01  0:00:01 --:--:--   192{   "{\n        \"userId\": 100,\n        \"id\": 100,\n        \"title\": \"main title\",\n        \"body\": \"main body\"\n    }": "",   "id": 101 } 

Java Code for the same is as follows:

OkHttpClient client = new OkHttpClient();  MediaType mediaType = MediaType.parse("application/octet-stream"); RequestBody body = RequestBody.create(mediaType, "{\n        \"userId\": 100,\n        \"id\": 100,\n        \"title\": \"main title\",\n        \"body\": \"main body\"\n    }"); Request request = new Request.Builder()   .url("https://jsonplaceholder.typicode.com/posts")   .post(body)   .addHeader("cache-control", "no-cache")   .addHeader("postman-token", "e11ce033-931a-0419-4903-ab860261a91a")   .build();  Response response = client.newCall(request).execute(); 

Another example of calling REST POST call with data ..

User user = new User(); user.setFirstName("john"); user.setLastName("Maclane");  ResteasyClient client = new ResteasyClientBuilder().build(); ResteasyWebTarget target = client.target("URL"); Response response = target.request().post(Entity.entity(user, <MEDIATYPE>)); //Read output in string format System.out.println(response.getStatus()); response.close();  

Here is the what your code looks like when I update it with my endpoints and payload.

import java.io.BufferedReader; import java.io.InputStreamReader; import java.io.OutputStreamWriter; import java.net.HttpURLConnection; import java.net.URL; import java.util.Arrays;  public class TestClass {      public static final String POST_URL = "https://jsonplaceholder.typicode.com/posts";      public static final String POST_DATA = "{\"userId\": 100,\"id\": 100,\"title\": \"main title\",\"body\": \"main body\"}";      public static void main(String[] args) throws Exception {         String[] details = {};         System.out.println(Arrays.toString(details));          URL line_api_url = new URL(POST_URL);         String payload = POST_DATA;          HttpURLConnection linec = (HttpURLConnection) line_api_url                 .openConnection();         linec.setDoInput(true);         linec.setDoOutput(true);         linec.setRequestMethod("POST");         linec.setRequestProperty("Content-Type", "application/json");         linec.setRequestProperty("Authorization", "Bearer "                 + "1djCb/mXV+KtryMxr6i1bXw");          OutputStreamWriter writer = new OutputStreamWriter(                 linec.getOutputStream(), "UTF-8");         writer.write(payload);          BufferedReader in = new BufferedReader(new InputStreamReader(                 linec.getInputStream()));         String inputLine;          while ((inputLine = in.readLine()) != null)             System.out.println(inputLine);         in.close();     } } 

In nutshell, check the API documentation and ensure the request payload is of correct format as 400 means BAD REQUEST.

Answers 2

It’s a 400 error, which means Bad Request. Please check this link below.

How to find out specifics of 400 Http error in Java?

Answers 3

OutputStreamWriter will buffer the output. After this line in your code:

writer.write(payload); 

add this line

writer.flush(); 

I would expect that to fix your problem.

Answers 4

Though this may not help you with precisely an existing HTTP call using the traditional HttpURLConnection. Yet an interesting way to achieve this in recent times is to use HTTP/2 Client and try out the latest introduced incubator module jdk.incubator.http form Java 9.

An easy way(quick-start) of mocking a POST call using the same is as follows :

  1. Create a within your project and define module-info.java as:

    module http.trial {      requires jdk.incubator.httpclient; } 
  2. Within the module create a package and a class named HttpPost with following content:

    import jdk.incubator.http.HttpRequest; import jdk.incubator.http.HttpClient; import jdk.incubator.http.HttpResponse; import java.io.IOException; import java.net.URI; import java.net.URISyntaxException;  public class HttpPost {    public static void main(String[] args) {      // Request builder     URI uri = null;     try {         uri = new URI("https://ads.line.me/api/v1.0/authority_delegations/get");     } catch (URISyntaxException e) {         e.printStackTrace();     }     HttpRequest.BodyProcessor bodyProcessor = HttpRequest.BodyProcessor.fromString("{\"operands\":[]}");     HttpRequest request = HttpRequest.newBuilder().uri(uri)                     .header("Content-Type", "application/json")                     .header("Authorization", "Bearer 1djCb/mXV+KtryMxr6i1bXw")                     .POST(bodyProcessor)                     .build();      // Client     HttpClient httpClient = HttpClient.newBuilder().followRedirects(HttpClient.Redirect.ALWAYS).build();     System.out.println(httpClient.version());      // Response builder     HttpResponse response = null;     try {         response = httpClient.send(request, HttpResponse.BodyHandler.asString());     } catch (IOException | InterruptedException e) {         e.printStackTrace();     }      System.out.println("StatusCode = " + response.statusCode());     System.out.println("Response = " + response.body().toString());   } } 

Answers 5

Thanks everyone for your help. I used below code to make it work.

public JSONObject send_post() {     HttpClient httpClient = HttpClientBuilder.create().build();     JSONObject jsonObject = null;      try {          HttpPost request = new HttpPost(this.URL + this.object_type + this.request_type);         StringEntity params = null;         if (this.request_type.equals("/get")) {             params = new StringEntity("{\"accountId\":\"5514\"}");         } else if (this.request_type.equals("/set")) {             // params = new             // StringEntity("{\"accountId\":\"5514\",\"operands\":[{\"id\":40151,\"name\":\"ddddd\"}]}");             String output = String.format("{\"accountId\":\"5514\",\"operands\":[{\"id\":%s,\"name\":\"%s\"}]}",                     this.params[0], this.params[1]);             if (this.params[1].equals("OptimizationOff")) {                 output = String.format(                         "{\"accountId\":\"5514\",\"operands\":[{\"id\":%s,\"bidOptimizationType\":\"%s\"}]}",                         this.params[0], "NONE");             } else if (this.params[1].equals("OptimizationOn")) {                 output = String.format(                         "{\"accountId\":\"5514\",\"operands\":[{\"id\":%s,\"bidOptimizationType\":\"%s\",\"bidOptimizationGoal\":\"%s\"}]}",                         this.params[0], this.params[2], this.params[3]);             }             if (object_type.equals("/ads")) {                 output = String.format("{\"accountId\":\"5514\",\"operands\":[{\"id\":%s,\"bidAmount\":\"%s\"}]}",                         this.params[0], this.params[1]);             }             params = new StringEntity(output);         }         request.addHeader("content-type", "application/json");         request.addHeader("Authorization", "Bearer " + this.Access_Token);         request.setEntity(params);          HttpResponse response = httpClient.execute(request);          BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));         StringBuffer result = new StringBuffer();         String line = "";         while ((line = rd.readLine()) != null) {             result.append(line);         }          System.out.println("API Resonse :"+result.toString());         jsonObject = new JSONObject(result.toString());      } catch (Exception ex) {          ex.printStackTrace();      } finally {      }     return jsonObject;  } 
Read More

Friday, July 14, 2017

How to put Cookie session id into volley request?

Leave a Comment

So, i have this code to make a POST request with volley:

public class MainActivity extends AppCompatActivity {   Button btnSearch;  ProgressDialog loadingDialog;  ListView lvResult;  String session_id;  RequestQueue queue;  MyCookieManager myCookieManager;   @Override  protected void onCreate(Bundle savedInstanceState) {   super.onCreate(savedInstanceState);   setContentView(R.layout.activity_main);    btnSearch = (Button) findViewById(R.id.btnSearch);   lvResult = (ListView) findViewById(R.id.lvResult);   loadingDialog = new ProgressDialog(MainActivity.this);   loadingDialog.setMessage("Wait.\nLoading...");   loadingDialog.setCancelable(false);   myCookieManager = new MyCookieManager();    requestCookie(); //FIRST CALL TO GET SESSION ID    btnSearch.setOnClickListener(new View.OnClickListener() {    @Override    public void onClick(View view) {     showLoading();     requestWithSomeHttpHeaders(); //CALL TO MAKE THE REQUEST WITH VALID SESSION ID    }   });   }   public void requestCookie() {   queue = Volley.newRequestQueue(this);   String url = "http://bid.cbf.com.br/a/bid/carregar/json/";    StringRequest postRequest = new StringRequest(Request.Method.POST, url,    new Response.Listener < String > () {     @Override     public void onResponse(String response) {      //      String x = myCookieManager.getCookieValue();     }    },    new Response.ErrorListener() {     @Override     public void onErrorResponse(VolleyError error) {      Log.d("ERRO", "Erro => " + error.toString());      hideLoading();     }    }   ) {    @Override    public byte[] getBody() throws AuthFailureError {     String httpPostBody = "uf=PE&dt_pesquisa=23/05/2017&tp_contrato=TODOS&n_atleta=&codigo_clube=&exercicio=";     return httpPostBody.getBytes();    }     @Override    public Map < String, String > getHeaders() throws AuthFailureError {     Map < String, String > params = new HashMap < String, String > ();     params.put("User-Agent", "Mozilla/5.0");     params.put("Accept", "application/json, text/javascript, */*; q=0.01");     params.put("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");     //params.put("Set-Cookie", session_id);// + " _ga=GA1.3.1300076726.1496455105; _gid=GA1.3.1624400465.1496455105; _gat=1; _gali=AguardeButton");     //"PHPSESSID=ra0nbm0l22gsnl6s4jo0qkqci1");     return params;    }     protected Response < String > parseNetworkResponse(NetworkResponse response) {     try {      String jsonString = new String(response.data, HttpHeaderParser.parseCharset(response.headers));      String header_response = String.valueOf(response.headers.values());      int index1 = header_response.indexOf("PHPSESSID=");      int index2 = header_response.indexOf("; path");      //Log.e(Utils.tag, "error is : " + index1 + "::" + index2);      session_id = header_response.substring(index1, index2);       return Response.success(jsonString, HttpHeaderParser.parseCacheHeaders(response));     } catch (UnsupportedEncodingException e) {      return Response.error(new ParseError(e));     }    }   };    queue.add(postRequest);  }   public void requestWithSomeHttpHeaders() {   queue = Volley.newRequestQueue(this);   String url = "http://bid.cbf.com.br/a/bid/carregar/json/";    StringRequest postRequest = new StringRequest(Request.Method.POST, url,    new Response.Listener < String > () {     @Override     public void onResponse(String response) {      Log.d("Response", response);      String x = myCookieManager.getCookieValue();      String status = "";       try {       JSONObject resultObject = new JSONObject(response);       Log.d("JSON RESULT =>", resultObject.toString());      } catch (JSONException e) {       Toast.makeText(MainActivity.this, "Request Error", Toast.LENGTH_SHORT).show();       e.printStackTrace();      }       hideLoading();     }    },    new Response.ErrorListener() {     @Override     public void onErrorResponse(VolleyError error) {      Log.d("ERROR", "Error => " + error.toString());      hideLoading();     }    }   ) {    @Override    public byte[] getBody() throws AuthFailureError {     String httpPostBody = "uf=PE&dt_pesquisa=23/05/2017&tp_contrato=TODOS&n_atleta=&codigo_clube=&exercicio=";     return httpPostBody.getBytes();    }     @Override    public Map < String, String > getHeaders() throws AuthFailureError {     Map < String, String > params = new HashMap < String, String > ();     params.put("User-Agent", "Mozilla/5.0");     params.put("Accept", "application/json, text/javascript, */*; q=0.01");     params.put("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");     params.put("Cookie", /*myCookieManager.getCookieValue()*/ session_id + "; _ga=GA1.3.1300076726.1496455105; _gid=GA1.3.1624400465.1496455105; _gat=1; _gali=AguardeButton");     return params;    }   };    queue.add(postRequest);  }   private void showLoading() {   runOnUiThread(new Runnable() {    @Override    public void run() {     if (!loadingDialog.isShowing())      loadingDialog.show();    }   });  }   private void hideLoading() {   runOnUiThread(new Runnable() {    @Override    public void run() {     if (loadingDialog.isShowing())      loadingDialog.dismiss();    }   });  } } 

If I send a valid cookie ID this return a valid JSON object else a empty object.

I tried (unsuccessfully) to set default cookie handles like

CookieManager manager = new CookieManager(); CookieHandler.setDefault(manager);

but I get a empty object.

How to put a valid cookie session ID to post request?

3 Answers

Answers 1

So the problem was getting a valid cookie. My mistake was to get it from the POST request itself. I kept the same working principle, getting the cookie when I started the application but using GET instead of POST and calling the root of the URL instead of the address where I get the JSON. My solution looked like this:

public void requestCookie() {   queue = Volley.newRequestQueue(this);   String url = "http://bid.cbf.com.br/";    StringRequest getRequest = new StringRequest(Request.Method.GET, url,    new Response.Listener < String > () {     @Override     public void onResponse(String response) {      String x = myCookieManager.getCookieValue();     }    },    new Response.ErrorListener() {     @Override     public void onErrorResponse(VolleyError error) {      Log.d("ERROR", "Error => " + error.toString());      hideLoading();     }    }   ) {       protected Response < String > parseNetworkResponse(NetworkResponse response) {     try {      String jsonString = new String(response.data, HttpHeaderParser.parseCharset(response.headers));      String header_response = String.valueOf(response.headers.values());      int index1 = header_response.indexOf("PHPSESSID=");      int index2 = header_response.indexOf("; path");      //Log.e(Utils.tag, "error is : " + index1 + "::" + index2);      session_id = header_response.substring(index1, index2);       return Response.success(jsonString, HttpHeaderParser.parseCacheHeaders(response));     } catch (UnsupportedEncodingException e) {      return Response.error(new ParseError(e));     }    }   };    queue.add(getRequest);  } 

Answers 2

You getting cookie (Session id) in Login response, Save cookie in sharedpreference or other db, and use that to send in request.

for getting cookie from login request

 CustomStringRequest stringRequest = new CustomStringRequest(Request.Method.POST, SIGN_IN_URL,                     new Response.Listener<CustomStringRequest.ResponseM>() {                         @Override                         public void onResponse(CustomStringRequest.ResponseM result) {                              CookieManager cookieManage = new CookieManager();                             CookieHandler.setDefault(cookieManage);                              progressDialog.hide();                             try {                                 //From here you will get headers                                 String sessionId = result.headers.get("Set-Cookie");                                 String responseString = result.response;                                  Log.e("session", sessionId);                                 Log.e("responseString", responseString);                                  JSONObject object = new JSONObject(responseString); 

CustomStringRequest class

public class CustomStringRequest extends Request<CustomStringRequest.ResponseM> {       private Response.Listener<CustomStringRequest.ResponseM> mListener;      public CustomStringRequest(int method, String url, Response.Listener<CustomStringRequest.ResponseM> responseListener, Response.ErrorListener listener) {         super(method, url, listener);         this.mListener = responseListener;     }       @Override     protected void deliverResponse(ResponseM response) {         this.mListener.onResponse(response);     }      @Override     protected Response<ResponseM> parseNetworkResponse(NetworkResponse response) {         String parsed;         try {             parsed = new String(response.data, HttpHeaderParser.parseCharset(response.headers));         } catch (UnsupportedEncodingException e) {             parsed = new String(response.data);         }          ResponseM responseM = new ResponseM();         responseM.headers = response.headers;         responseM.response = parsed;          return Response.success(responseM, HttpHeaderParser.parseCacheHeaders(response));     }       public static class ResponseM {         public Map<String, String> headers;         public String response;     }  } 

set cookie when add request to server..

                @Override                 public Map<String, String> getHeaders() throws AuthFailureError {                     Map<String, String> headers = new HashMap<String, String>();                      String session=sharedPreferences.getString("sessionId","");                     headers.put("Cookie",session);                     return headers;                 } 

Answers 3

Using cookies with Android volley library

Request class:

public class StringRequest extends com.android.volley.toolbox.StringRequest {      private final Map<String, String> _params;      /**      * @param method      * @param url      * @param params      *            A {@link HashMap} to post with the request. Null is allowed      *            and indicates no parameters will be posted along with request.      * @param listener      * @param errorListener      */     public StringRequest(int method, String url, Map<String, String> params, Listener<String> listener,             ErrorListener errorListener) {         super(method, url, listener, errorListener);          _params = params;     }      @Override     protected Map<String, String> getParams() {         return _params;     }      /* (non-Javadoc)      * @see com.android.volley.toolbox.StringRequest#parseNetworkResponse(com.android.volley.NetworkResponse)      */     @Override     protected Response<String> parseNetworkResponse(NetworkResponse response) {         // since we don't know which of the two underlying network vehicles         // will Volley use, we have to handle and store session cookies manually         MyApp.get().checkSessionCookie(response.headers);          return super.parseNetworkResponse(response);     }      /* (non-Javadoc)      * @see com.android.volley.Request#getHeaders()      */     @Override     public Map<String, String> getHeaders() throws AuthFailureError {         Map<String, String> headers = super.getHeaders();          if (headers == null                 || headers.equals(Collections.emptyMap())) {             headers = new HashMap<String, String>();         }          MyApp.get().addSessionCookie(headers);          return headers;     } } 

MyApp:

public class MyApp extends Application {     private static final String SET_COOKIE_KEY = "Set-Cookie";     private static final String COOKIE_KEY = "Cookie";     private static final String SESSION_COOKIE = "sessionid";      private static MyApp _instance;   private RequestQueue _requestQueue;   private SharedPreferences _preferences;      public static MyApp get() {         return _instance;     }      @Override     public void onCreate() {         super.onCreate();         _instance = this;             _preferences = PreferenceManager.getDefaultSharedPreferences(this);         _requestQueue = Volley.newRequestQueue(this);     }      public RequestQueue getRequestQueue() {         return _requestQueue;     }       /**      * Checks the response headers for session cookie and saves it      * if it finds it.      * @param headers Response Headers.      */     public final void checkSessionCookie(Map<String, String> headers) {         if (headers.containsKey(SET_COOKIE_KEY)                 && headers.get(SET_COOKIE_KEY).startsWith(SESSION_COOKIE)) {                 String cookie = headers.get(SET_COOKIE_KEY);                 if (cookie.length() > 0) {                     String[] splitCookie = cookie.split(";");                     String[] splitSessionId = splitCookie[0].split("=");                     cookie = splitSessionId[1];                     Editor prefEditor = _preferences.edit();                     prefEditor.putString(SESSION_COOKIE, cookie);                     prefEditor.commit();                 }             }     }      /**      * Adds session cookie to headers if exists.      * @param headers      */     public final void addSessionCookie(Map<String, String> headers) {         String sessionId = _preferences.getString(SESSION_COOKIE, "");         if (sessionId.length() > 0) {             StringBuilder builder = new StringBuilder();             builder.append(SESSION_COOKIE);             builder.append("=");             builder.append(sessionId);             if (headers.containsKey(COOKIE_KEY)) {                 builder.append("; ");                 builder.append(headers.get(COOKIE_KEY));             }             headers.put(COOKIE_KEY, builder.toString());         }     }  } 
Read More

Saturday, June 10, 2017

How can I start / stop a request stream with node.js?

Leave a Comment

I have

return request({   method: "POST",   url: response.stream.url,   json: true,   forever: true,   body: {     sessionid: response.stream.sessionid,     symbols: symbolParams   } }).on("data", onData) 

That gives me data whenever it comes through. But what if I want to STOP listening on that stream, how can I do that?

3 Answers

Answers 1

It seems to implement Readable Stream interface, so pause() and resume() work.

const request = require('request'); const req = request('http://google.com')   .on('data', data => {        console.log(data);         req.pause();      }); 

Answers 2

You can use https://nodejs.org/api/events.html#events_emitter_removelistener_eventname_listener

const req = request({     method: "POST",     url: response.stream.url,     json: true,     forever: true,     body: {         sessionid: response.stream.sessionid,         symbols: symbolParams     } }).on("data", onData); ... req.removeEventListener('data', onData); 

Answers 3

You can also abort it:

var req = request('http://google.com')... ... req.abort(); 
Read More

Thursday, June 8, 2017

How to put Cookie session id into volley request?

Leave a Comment

So, i have this code to make a POST request with volley:

public void myRequest() {  RequestQueue queue = Volley.newRequestQueue(this);  String url = "http://bid.cbf.com.br/a/bid/carregar/json/";  StringRequest postRequest = new StringRequest(Request.Method.POST, url,   new Response.Listener <String> () {    @Override    public void onResponse(String response) {     Log.d("Response", response);    }   },   new Response.ErrorListener() {    @Override    public void onErrorResponse(VolleyError error) {     Log.d("ERROR", "Error => " + error.toString());    }   }  ) {   @Override   public byte[] getBody() throws AuthFailureError {    String httpPostBody = "uf=PE&dt_pesquisa=23%2F05%2F2017&tp_contrato=TODOS&n_atleta=&codigo_clube=&exercicio=";    return httpPostBody.getBytes();   }    @Override   public Map <String, String> getHeaders() throws AuthFailureError {    Map <String, String> params = new HashMap <String, String> ();    params.put("User-Agent", "Mozilla/5.0");    params.put("Accept", "application/json, text/javascript, */*; q=0.01");    params.put("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");    //params.put("Cookie", "PHPSESSID=ra0nbm0l22gsnl6s4jo0qkqci1"); PROBLEM HERE    return params;   }  };   queue.add(postRequest); } 

If I send a valid cookie ID this return a valid JSON object else a empty object.

I tried (unsuccessfully) to set default cookie handles like

CookieManager manager = new CookieManager(); CookieHandler.setDefault(manager);

but I get a empty object.

How to put a valid cookie session ID to post request?

0 Answers

Read More

Sunday, April 30, 2017

Relative form action removes 4th grade subdomain

Leave a Comment

I have issue with HTML form which is pointing its action to: /index.php?something=x

So its looks like

<form action="/index.php?something=x" method="POST"> 

I have production of application running on subdomain xx.example.com

When i submit form, everything works well, request is going to:

xx.example.com/index.php?something=x

But on development environment i have 4th grade url. Example: yy.xx.example.com

When i submit form on dev environment request is not going to https://yy.xx.example.com/index.php?something=x

but url is without yy => https://xx.example.com/index.php?something=x and it is wrong.

Any suggestions?

2 Answers

Answers 1

It's not problem in URL, you can have domain or sub domain as you like.
it can be, as you said 4th grade URL or longer "https://zz.yy.xx.example.com/".
Tested in my localhost xampp and on real server with some test sub domain.

Try to fix your code, you have two action fields

action="POST" 

Replace second "action" with "method". It should be like this:

<form action="/index.php?something=x" method="POST"> 

Answers 2

To check, what is happening with your site, I have created two subdomains on the domain http://techdeft.com namely http://xx.techdeft.com and http://yy.xx.techdeft.com.

Now when I created a simple form on these sub-domains and actions set to, what you have mentioned in the question, I found that everything works fine with these sub-domains. You can check here http://xx.techdeft.com and http://yy.xx.techdeft.com.

here, is the possible solution to your problem-

<form action="http://<?php echo $_SERVER['SERVER_NAME'];?>/index.php?something=x" method="POST"> 

Using this your problem must be solved. Do let me know, have it worked for you? Thanks

Read More

Monday, March 20, 2017

Express.js http call inside route doesn't update variable

Leave a Comment

I'm building a REST api on top of express.js. I am having trouble updating variables inside my routes.

Example:

I'm calling app.get("/wp/page/create/:id", function(req, res)

Inside this route I start by calling a http request using request-promise library. The response of this call I use in a nested http call.

I use a global variable for the headers for the nested call, and it's to the header a i need to make changes by using the etag variable.

Code:

global.postHeaders = headers; postHeaders['X-HTTP-Method'] = "MERGE"; postHeaders['Content-Type'] = 'application/json;odata=verbose'; postHeaders['X-RequestDigest'] = spContext;  request.get({ url: "xxx", headers: headers, json: true }).then(function(response) {     var etag = response.d.__metadata.etag     postHeaders['If-Match'] = etag;      request.post({        url: "xxx",        type: "POST",        body: data,        headers: postHeaders,        json: true        }).then(function(data) {           res.send(data).end()           console.log("All done!");       }) }) 

When i start the server up and enter the route everything works fine. When i when try to hit it again the etag variables is still the same, even though it should be updated.

If I restart the server it works the again on the first attempt but fails on the second/third.

Any idea what I am doing wrong?

2 Answers

Answers 1

I have resolved the issues. The simple solution was to clear the headers containing the variable.

global.postHeaders = headers; postHeaders['X-HTTP-Method'] = "MERGE"; postHeaders['Content-Type'] = 'application/json;odata=verbose'; postHeaders['X-RequestDigest'] = spContext;  request.get({  url: "xxx",  headers: headers,  json: true  }).then(function(response) { var etag = response.d.__metadata.etag postHeaders['If-Match'] = etag;     request.post({      url: "xxx",      type: "POST",      body: data,      headers: postHeaders,      json: true      }).then(function(data) {       postHeaders['If-Match'] = "";       res.send(data).end()       console.log("All done!");   }) }) 

Answers 2

postHeaders is a global variable. is headers in global.postHeaders = headers; also a global varaible ? Whatever you are trying to do here is grossly wrong. postHeaders variable will be shared across multiple request. so you will hit a scenario where postHeaders['If-Match'] value might be empty string or the etag .

Try this instead of the first line var postHeaders = Object.assign({}, headers);

Not sure what you are trying, but at-least this statement will subside the huge error in the code. This will create a new header object for each request.

Read More

Monday, March 28, 2016

curl url(custum port) html grab show blank on live site php

Leave a Comment

I want to grab html of URL with custom port(instead of 8080) i.e abc.com:1234 but its shows blank on live server but works fine on localhost

My Code:

     <?php ini_set("display_errors",1); error_reporting(E_ALL);                          ini_set('allow_url_fopen', true); ini_set('allow_url_include', true); ini_set('allow_url_include', 'on'); ?>                       <?php   //getSslPage("http://portquiz.net:8080");           getSslPage("http://portquiz.net:666");                           function getSslPage($url) {           $curl_connection =   curl_init($url);          //set options ///         curl_setopt($curl_connection, CURLOPT_CONNECTTIMEOUT, 3990);                          curl_setopt($curl_connection, CURLOPT_USERAGENT,   "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");                          curl_setopt($curl_connection, CURLOPT_HTTPHEADER, array(                              'Content-Type: text/xml',                             'Connection: Keep-Alive',                             'Keep-Alive: 300' ));                          curl_setopt($curl_connection, CURLOPT_RETURNTRANSFER, true);     curl_setopt($curl_connection,CURLOPT_SSL_VERIFYHOST,0);      curl_setopt($curl_connection, CURLOPT_SSL_VERIFYPEER, false);      curl_setopt($curl_connection, CURLOPT_FOLLOWLOCATION, 1);      curl_setopt($curl_connection, CURLOPT_TIMEOUT, 20); //old value 15               //set data to be posted              //curl_setopt($curl_connection, CURLOPT_POSTFIELDS, $post_string);              //perform our request               $result = curl_exec($curl_connection);               echo "<h1 style=color:blue>Output Result:</h1><br>";               echo htmlentities($result);                echo "<h1 style=color:blue>|Curl Exec Detail:</h1><br>";               //var_dump($result);               echo "<pre>";              print_r(curl_getinfo($curl_connection));                echo "</pre>";                echo curl_errno($curl_connection) . '<br/>';              echo curl_error($curl_connection) . '<br/>';                             } exit; ?> 

Output on Localhost:

Output Result:  <html> <head> <title>Outgoing Port Tester</title> <style type="text/css"> body { font-family: sans-serif; font-size: 0.9em; } </style> </head> <body> <h1>Outgoing port tester</h1> This server listens on all TCP ports, allowing you to test any outbound TCP port. <p> You have reached this page on port <b>666</b>.<br/> </p> Your network allows you to use this port. (Assuming that your network is not doing advanced traffic filtering.) <p> Network service: unknown<br/> Your outgoing IP: 39.32.94.166</p> <h2>Test a port using a command</h2> <pre> $ telnet portquiz.net 666 Trying ... Connected to portquiz.net. Escape character is '^]'. </pre> <pre> $ nc -v portquiz.net 666 Connection to portquiz.net 666 port [tcp/daytime] succeeded! </pre> <pre> $ curl portquiz.net:666 Port 666 test successful! Your IP: 39.32.94.166</pre> <pre> $ wget -qO- portquiz.net:666 Port 666 test successful! Your IP: 39.32.94.166</pre> <pre> # For Windows PowerShell users PS C:\&gt; Test-NetConnection -InformationLevel detailed -ComputerName portquiz.net -Port 666</pre> <h2>Test a port using your browser</h2> <p> In your browser address bar: <strong>http://portquiz.net:XXXX</strong> </p> Examples: <br/> <a href="http://portquiz.net:8080">http://portquiz.net:8080</a> <br/> <a href="http://portquiz.net:8">http://portquiz.net:8</a> <br/> <a href="http://portquiz.net:666">http://portquiz.net:666</a> <br/> <p> I got complains that portquiz is not working on port 445. My hosting company OVH is probably blocking this port. Sorry about that. Feel free to contact them. See <a href="http://positon.org/is-ovh-blocking-port-445">my blog post</a> and <a href="https://forum.ovh.com/showthread.php/106901-OVH-bloque-le-port-445-vers-mon-serveur-d%C3%A9di%C3%A9">OVH forum post (french)</a>. </p> <p> Your browser can block network ports normally used for purposes other than Web browsing. In this case you should use the telnet or netcat commands to test the port. </p> <p> Please also note that this server uses some port for real services (22, 25), so testing with your browser on those ports will not work. </p> <p> <i>Contact/feedback:</i><br/> <img src="portquizm.png" /> </p> <p> <i>See also:</i><br/> <ul> <li> <a href="http://en.positon.org/post/An-outgoing-port-tester">Blog post on this topic</a> and <a href="http://positon.org/portquiz-net-how-it-works">How it works</a> </li> <li> <a href="http://www.firebind.com/">Firebind</a>, a commercial tester. <a href="http://www.firebind.com/clients/web/">javascript test</a> </li> <li> <a href="https://github.com/nhooyr/outPorts">outPorts</a>, a tiny program to test a range of ports using portquiz </li> </ul> </p> </body> </html> |Curl Exec Detail:  Array (     [url] => http://portquiz.net:666/     [content_type] => text/html     [http_code] => 200     [header_size] => 233     [request_size] => 184     [filetime] => -1     [ssl_verify_result] => 0     [redirect_count] => 0     [total_time] => 0.5     [namelookup_time] => 0     [connect_time] => 0.234     [pretransfer_time] => 0.234     [size_upload] => 0     [size_download] => 2694     [speed_download] => 5388     [speed_upload] => 0     [download_content_length] => 2694     [upload_content_length] => 0     [starttransfer_time] => 0.5     [redirect_time] => 0     [redirect_url] =>      [primary_ip] => 178.33.250.62     [certinfo] => Array         (         )      [primary_port] => 666     [local_ip] => 192.168.1.47     [local_port] => 54754 )  0 

Output on Live Site:(result show blank/Empty)

Output Result:  |Curl Exec Detail:  Array (     [url] => http://portquiz.net:666/     [content_type] =>      [http_code] => 0     [header_size] => 0     [request_size] => 0     [filetime] => -1     [ssl_verify_result] => 0     [redirect_count] => 0     [total_time] => 19.340628     [namelookup_time] => 0.117558     [connect_time] => 0     [pretransfer_time] => 0     [size_upload] => 0     [size_download] => 0     [speed_download] => 0     [speed_upload] => 0     [download_content_length] => -1     [upload_content_length] => -1     [starttransfer_time] => 0     [redirect_time] => 0     [redirect_url] =>      [primary_ip] =>      [certinfo] => Array         (         )      [primary_port] => 0     [local_ip] =>      [local_port] => 0 )  28 Connection timed out after 20001 milliseconds 

Please Help me i am working it from last 2 days but no success thanks

5 Answers

Answers 1

In the example you posted above i saw no curl_exec, so i added that.

This works for me:

<?php ini_set("display_errors", 1); error_reporting(E_ALL);  ini_set('allow_url_fopen', true); ini_set('allow_url_include', true); ini_set('allow_url_include', 'on'); ?>  <?php  //getSslPage("http://portquiz.net:8");  function getSslPage($url) {     $curl_connection = curl_init($url); //set options ///curl_setopt($curl_connection, CURLOPT_CONNECTTIMEOUT, 3990);      curl_setopt($curl_connection, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");      curl_setopt($curl_connection, CURLOPT_HTTPHEADER, array(         'Content-Type: text/xml',         'Connection: Keep-Alive',         'Keep-Alive: 300'));      curl_setopt($curl_connection, CURLOPT_RETURNTRANSFER, true);     curl_setopt($curl_connection, CURLOPT_SSL_VERIFYHOST, 0);     curl_setopt($curl_connection, CURLOPT_SSL_VERIFYPEER, false);     curl_setopt($curl_connection, CURLOPT_FOLLOWLOCATION, 1);     curl_setopt($curl_connection, CURLOPT_TIMEOUT, 20); //old value 15 //set data to be posted //curl_setopt($curl_connection, CURLOPT_POSTFIELDS, $post_string); //perform our request  $result = curl_exec($curl_connection);  echo "<h1 style=color:blue>Output Result:</h1><br>";  echo htmlentities($result);   echo "<h1 style=color:blue>|Curl Exec Detail:</h1><br>";  //var_dump($result);  echo "<pre>";     echo "</pre>";     echo curl_errno($curl_connection) . '<br/>';     echo curl_error($curl_connection) . '<br/>';      print_r(curl_exec($curl_connection));      print_r(curl_getinfo($curl_connection)); }  getSslPage("http://portquiz.net:666"); 

result:

</pre>0<br/><br/> <html> <head> <title>Outgoing Port Tester</title> <style type="text/css"> body {     font-family: sans-serif;     font-size: 0.9em; } </style>  </head>  <body> <h1>Outgoing port tester</h1>  This server listens on all TCP ports, allowing you to test any outbound TCP port.  <p> You have reached this page on port <b>666</b>.<br/> </p>  Your network allows you to use this port. (Assuming that your network is not doing advanced traffic filtering.)  <p> Network service: unknown<br/> Your outgoing IP: 131.228.182.254</p>  <h2>Test a port using a command</h2>  <pre> $ telnet portquiz.net 666  Trying ... Connected to portquiz.net. Escape character is '^]'. </pre> <pre> $ nc -v portquiz.net 666  Connection to portquiz.net 666 port [tcp/daytime] succeeded! </pre> <pre> $ curl portquiz.net:666  Port 666 test successful! Your IP: 131.228.182.254</pre> <pre> $ wget -qO- portquiz.net:666  Port 666 test successful! Your IP: 131.228.182.254</pre> <pre> # For Windows PowerShell users PS C:\&gt; Test-NetConnection -InformationLevel detailed -ComputerName portquiz.net -Port 666</pre>  <h2>Test a port using your browser</h2>  <p> In your browser address bar: <strong>http://portquiz.net:XXXX</strong> </p>  Examples: <br/> <a href="http://portquiz.net:8080">http://portquiz.net:8080</a> <br/> <a href="http://portquiz.net:8">http://portquiz.net:8</a> <br/> <a href="http://portquiz.net:666">http://portquiz.net:666</a> <br/>  <p> I got complains that portquiz is not working on port 445. My hosting company OVH is probably blocking this port. Sorry about that. Feel free to contact them. See <a href="http://positon.org/is-ovh-blocking-port-445">my blog post</a> and <a href="https://forum.ovh.com/showthread.php/106901-OVH-bloque-le-port-445-vers-mon-serveur-d%C3%A9di%C3%A9">OVH forum post (french)</a>. </p>  <p> Your browser can block network ports normally used for purposes other than Web browsing. In this case you should use the telnet or netcat commands to test the port. </p> <p> Please also note that this server uses some port for real services (22, 25), so testing with your browser on those ports will not work. </p>   <p> <i>Contact/feedback:</i><br/> <img src="portquizm.png" /> </p>  <p> <i>See also:</i><br/> <ul> <li> <a href="http://en.positon.org/post/An-outgoing-port-tester">Blog post on this topic</a> and <a href="http://positon.org/portquiz-net-how-it-works">How it works</a> </li> <li> <a href="http://www.firebind.com/">Firebind</a>, a commercial tester. <a href="http://www.firebind.com/clients/web/">javascript test</a> </li> <li> <a href="https://github.com/nhooyr/outPorts">outPorts</a>, a tiny program to test a range of ports using portquiz </li> </ul> </p>  </body>  </html> Array (     [url] => http://portquiz.net:666/     [content_type] => text/html     [http_code] => 200     [header_size] => 233     [request_size] => 184     [filetime] => -1     [ssl_verify_result] => 0     [redirect_count] => 0     [total_time] => 0.054183     [namelookup_time] => 0.004118     [connect_time] => 0.024217     [pretransfer_time] => 0.024252     [size_upload] => 0     [size_download] => 2703     [speed_download] => 49886     [speed_upload] => 0     [download_content_length] => 2703     [upload_content_length] => -1     [starttransfer_time] => 0.053941     [redirect_time] => 0     [redirect_url] =>      [primary_ip] => 178.33.250.62     [certinfo] => Array     (     )      [primary_port] => 666     [local_ip] => 10.223.128.174     [local_port] => 36696 ) 

Answers 2

The code seems fine to me, as a workaround try changing :

curl_setopt ( $curl_connection, CURLOPT_RETURNTRANSFER, true );

to

curl_setopt ( $curl_connection, CURLOPT_RETURNTRANSFER, 1 );

As described in the Return Values section of curl-exec PHP manual page: http://php.net/manual/function.curl-exec.php

You should enable the CURLOPT_FOLLOWLOCATION option for redirects but this would be a problem if your server is in safe_mode and/or open_basedir is in effect which can cause issues with curl as well.

Answers 3

The important message here is:

Connection timed out after 20001 milliseconds

It means your connection didn't go through to that host. The reason maybe a firewall or network misconfiguration on either end - your server or remote server.

The first thing that comes to my mind is port number (666) that is lower than 1024. On Unix systems only privileged users can listen to those ports. So if you are using regular user you could be unable to listen to it.

But if you are able to connect to it from your local machine the problem is probably is with your server. In order to check if it is configured to pass connections to any port you can try this simple command in your terminal:

$ telnet portquiz.net 666 Trying 178.33.250.62... Connected to portquiz.net. Escape character is '^]'. GET / 

GET / is HTTP request you need to enter from keyboard if/when you get the prompt. If you don't get the prompt and you get timeout message after some time - your host is configured to not pass connections to custom ports. This is often the issue with shared hosting.

In such a case you should do one of the following:

  • reconfigure your host to allow such connections
  • use some standard port: 80, 8080, 443, etc
  • switch your hosting provider to the one that doesn't have such restrictions.

EDIT: You can also do a simple check from your PHP script:

$fp = fsockopen("portquiz.net", 666); var_dump($fp); 

If result of var_dump is false or you get a timeout executing that page - the reason is connecting to random ports is not permitted on your hosting.

Answers 4

Note that curl_* functions in PHP are wrapper of curl program.

It means you can try same thing via command line.

If you can connect your live server via SSH or Telnet,

Try curl http://portquiz.net:666 -v and see what the real problem is.

Answers 5

Have you noticed this error "Connection timed out after 20001 milliseconds"

Increase your Connection time out time here add enough time instead of 20

curl_setopt($curl_connection, CURLOPT_TIMEOUT, 20); 
Read More