How do I get inclusive sample that aggregates children?

In VS 2015, I have run CPU Usage Profile and I see a function (FunctionA) that has Inclusive Samples of 10%. This function is called from several other functions (ParentFunctions 1,2&3).

When I look at Inclusive Samples of the 3 parent functions, their combined inclusive sample is way less than the IS of the child FunctionB!

How can I tell which Parent is causing the most usage? I am used to using a previous profiler where the Parent "CPU Time" was an aggregate of its children. Why do Inclusive Samples NOT work like this? How can I switch to something that gives me actual time? Or, is there some other I am able to tell which is the naughty parent?

1 Answers

Answers 1

You have the usage profile but what people do not know is that Visual Studio 2015 has a sampling profiling method:

The sampling profiling method of the Visual Studio Profiling Tools interrupts the computer processor at set intervals and collects the function call stack. A call stack is a dynamic structure that stores information about the functions that are executing on the processor.

But do note the requirements:

Visual Studio Ultimate, Visual Studio Premium, Visual Studio Professional

It will give you also the inclusive samples:

The total number of samples that are collected during the execution of the target function.

This includes samples that are collected during the direct execution of the function code and samples that are collected during the execution of child functions that are called by the target function.

The steps to produce it:

• Open the solution in Visual Studio. Set the configuration to Release. (Find the Solution Configurations box on the toolbar, which is set to Debug by default. Change it to Release.)
• On the Debug menu, select Profiler, and then select Performance Profiler.
• Check the Performance Wizard option, and click Start.
• Check the CPU Sampling (recommended) option and click Finish.

Please do note:

• After you finish running the application, the Summary view of the profiling data appears in the main Visual Studio window and an icon for the new session appears in the Performance Explorer window.

• If you are not an administrator on the computer that you are using, you should run Visual Studio as an administrator while you are using the profiler. (Right-click the Visual Studio application icon, and then click Run as administrator.

use this guide for reference.

Read More

Horrible performance when upgrading from node v6.7.0 to v8.11.1

I have been spending the past few days investigating poor performance for an websocket graphql api application when upgrading node from v6.x to v8.x.

I have taken lots of flame graphs but I cannot figure out where the bottleneck is. Does anyone know what ___kdebug_trace_string (in c++) is? It appears to be taking significantly more time in my application after the upgrade.

Check out this flame graph:

Also check out these profile logs:

node v8.x profile log (slow): https://pastebin.com/2W65BZC8

node v6.x profile log: https://pastebin.com/BL4kM7B7

Thanks ahead!

1 Answers

Answers 1

kdebug_trace_string is a syscall that has been added to XNU in october 2015 for iOS 9 and OS X 10.11.

It's part of kdebug, the main XNU built-in debugging facility. When reading kdebug_trace.c, I found the following note in the comments :

Note that the userspace API is chosing to optimize fastpath, non-error performance by eliding validation of each debugid. This means that error cases which could have been caught in userspace will make a syscall before returning with the correct error code. This tradeoff in performance is intentional.

It explains why ___kdebug_trace_string is taking so much place on your flame graph.

---

This is just a guess and all of this is wrong if you're not using an Apple computer, but then, if it's wrong, I really want to know what's causing this mess.

---

Assuming I'm right, if kdebug_trace_string is called, so it means that node runs some kind of debugging utility. I downloaded node-v8.11.1-darwin-x64, and found the following line in node/config.gypi :

 'node_use_dtrace': 'true', 

So node v8.11.1 uses dtrace.
Then, looking at this line in osx/src/dtrace/libdtrace/dt_open.c, we can assume that dtrace uses kdebug_trace_string

So to fix this, one would want to prevent node from using dtrace. According to this answer, " When Node starts up, the .gypi is loaded like any other settings file.". So maybe you should juste set node_use_dtrace to false

BUT

I don't understand why you didn't face the same problem with node v6.7.0 :

• In node-v6.7.0-darwin-x64, node_use_dtrace is set to true too
• Node v6.7 approx. release date : 2016-09-28
• OS X 10.11 approx release date : 2015-09-06

Could you tell me the value of node_use_dtrace for your two versions of node ?

Hope it helps, and hope I'm right,
Best regards

Read More

time of all child processes doesn't come close to time of parent process

Today is my first day using R's profvis::profvis function and it seems very useful.

It seems to me, though, that the sum of the time taken by all the children of a process should approximately sum to the time of the process itself.

But I've observed that this isn't true with stats::glm. (See attached picture, a screenshot from the profiling pane of RStudio.)

Where have I gone wrong?

EDIT: I am an idiot -- these numbers add up to 1180, very close to their stated sum of 1200. Please ignore this question! (Can't delete it.)

0 Answers

Read More

Monitor maximum memory consumption in Node.js process

I'm looking for a cross-platform way to reliably monitor maximum memory consumption in Node.js process, regardless of whether there are leaks or not.

The processes in my case are both real applications and synthetic tests.

I would expect it to work like

process.on('exit', () => {   console.log('Max memory consumption: ' + ...); }); 

It was possible to trace memory consumption somehow with node --trace_gc ..., but this resulted in output that is hard to read (and probably hard to analyze programmatically). Also, GC didn't occur when a script ended too fast, even if RAM usage was substantial.

From what I have seen on the subject, usually memwatch is suggested for that, like:

require('memwatch-next').on('stats', stats => {   console.log('Max memory consumption: ' + stats.max); }); 

But in my case it triggered only when GC already happened or didn't trigger at all, so it was useless for determining RAM consumption peaks.

I would prefer to avoid GUI tools like node-inspector if possible.

Can this maximum memory consumption be reliably retrieved as a number from the application itself or CLI alone, cross-platform?

2 Answers

Answers 1

You could use Node.js process.memoryUsage() method to get memory usage stat:

The process.memoryUsage() method returns an object describing the memory usage of the Node.js process measured in bytes.

It returns the object of the following format:

{   rss: 4935680,       // Resident Set Size   heapTotal: 1826816, // Total Size of the Heap   heapUsed: 650472,   // Heap actually Used   external: 49879     // memory usage of C++ objects bound to JavaScript objects managed by V8 } 

To get the maximum memory consumption in Node.js process, process.nextTick method could be used. process.nextTick() method adds the callback to the next tick queue. Once the current turn of the event loop turn runs to completion, all callbacks currently in the next tick queue will be called.

let _maxMemoryConsumption; let _dtOfMaxMemoryConsumption;  process.nextTick(() => {   let memUsage = process.memoryUsage();   if (memUsage.rss > _maxMemoryConsumption) {     _maxMemoryConsumption = memUsage.rss;     _dtOfMaxMemoryConsumption = new Date();   } });  process.on('exit', () => {   console.log(`Max memory consumption: ${_maxMemoryConsumption} at ${_dtOfMaxMemoryConsumption}`); }); 

Answers 2

If you try to benchmark the process from within itself, you will have distorted memory usage values. (if you want more information about this, just comment)

---

This is a little (crossplatform) tool I coded for checking the memory usage of another process, it spawns an independent process and watches every 100ms for memory usage in order to find the highest peak, outputs everytime a new peak is found and stops once child has ended.

It uses pidusage which is a crossplatform process ( cpu % and ) memory usage of a PID

Allows customization of the spawn (arguments to be passed along with the spawn) [could be updated for command line usage]

It will also work with any node binary name, since it will reuse the one used to start this tool.

'use strict' const UI = {};  var ñ = "    " const pusage = require('pidusage'); //:Setup the 'cmd' array to be the file and arguments to be used const ANALYSIS = {cmd:['child.js']} ANALYSIS.child = require('child_process').spawn(  process.argv[0], // reuse to work with the same binary name used to run this (node|nodejs|...)  ANALYSIS.cmd,   // array with filePath & arguments to spawn for this analisis  { //So the child_process doesn't behave like a child    detached:true,        stdio:['ignore'],    env:null  } ); //:The Analysis DoAnalysis(ANALYSIS.child.pid); ANALYSIS.child.unref() var memPeak = 0; function PIDStat(){  pusage.stat(ANALYSIS.pid, function(err, stat) {   if(err){ CheckError(err) }else{    if(stat.memory > memPeak){memPeak=stat.memory;PrintStat()}    setTimeout(PIDStat,100); pusage.unmonitor(process.pid)   }  }) } //:UI (just for display) function DoAnalysis(PID){  var s = '═'.repeat(ANALYSIS.cmd[0].toString().length)  ANALYSIS.pid = PID;  UI.top = '╒═'+s+'═╕'  UI.mid = '│ '+ANALYSIS.cmd[0]+' │'  UI.bot = '╘═'+s+'═╛'  console.log(UI.x);  PIDStat() } function PrintStat(){  console.clear()  console.log('\n',UI.top,'\n',UI.mid,'PEAK MEM. :',memPeak,'\n',UI.bot) } function CheckError(e){  switch(e.code){   case "ENOENT": console.log("              [the analysis ended]\n"); break;   default: console.log("[/!\\ error]\n",e); break  } } 

Will produce the following output :

 ╒══════════╕   │ child.js │ PEAK MEM. : 28737536   ╘══════════╛               [the analysis ended] 

This tool prevents you from adding bloat to the code of the process you actually want to benchmark, so this way you wont get different memory usage values, since your bloat/benchmarking code will also add memory usage to that process.

Read More

swift_dynamiccast taking too much in processing and affecting performance in iOS

I am doing a lot of computing in parsing and creating some combination from json data coming from server. The entire process takes a lot of time and majorly I have fixed code related issue but at one place time profiler shows the time taken by one particular call which I cannot figure out.

I have a lot of casting going on in my processing. It creates a lot of objects of type FlightFare and I create it from a dictionary.

So the conveninece init looks like below, how can I avoid it..?

convenience init (dictionary: [String:AnyObject]) {     self.init()     refundType = dictionary["rt"] as! String     if let unwrappedScore = dictionary["r"] as? Double {         score = unwrappedScore     }     if let unwrappedValue = dictionary["t"] as? Int {         taxes = unwrappedValue     }     if let unwrappedValue = dictionary["bf"] as? Int {         baseFare = unwrappedValue     }     if let unwrappedValue = dictionary["f"] as? Int {         fee = unwrappedValue     }     if let unwrappedValue = dictionary["d"] as? Int {         discount = unwrappedValue     }     if let unwrappedValue = dictionary["tf"] as? Int {         fare = unwrappedValue     }     if let unwrappedValue = dictionary["ttf"] as? Int {         totalFare = unwrappedValue     }     if let unwrappedValue = dictionary["hbo"] as? Bool {         hbo = unwrappedValue     }     providerKey = dictionary["pk"] as? String     hbf = dictionary["hbf"] as? [String] } 

4 Answers

Answers 1

Sounds like the tax paid when doing things like value as? String. You left that node collapsed in the profile, so I can only give general advice on avoiding this: Ensure you aren't repeatedly processing the same data in its raw form. Cast once as part of migrating to a typed intermediate form.

Minimizing type casts is something that the various Swift JSON libraries, like Freddy and SwiftyJSON, attempt to guarantee in their parsing of JSON.

If the structure of your JSON data is similarly uncertain - is this node a string or an object or just null? - then your code runs into and must address this issue at that level, too.

ETA: If the cost of the casts is noticeably charged to your convenience init (you should verify this using the profile), then that method would indeed be a problem. You can avoid it by using a library that parses JSON into a typed representation to begin with, such as Freddy mentioned earlier. This replaces casting in your code with enum case matching within the library.

It's often useful to check the box in the profile options that tells it not to show you framework code, so you can easily see which of your methods that you directly control is taking time. Then you can drill in in the profile to get per-line cost statistics to focus your optimization.

Answers 2

You could write the init() a little differently to separate unwrapping form type casting and gain around 40% speed

       for (key,value) in dictionary        {           switch key           {              case "rt"  : refundType  = value  as! String              case "r"   : score       = value  as! Double              case "bf"  : baseFare    = value  as! Int              case "f"   : fee         = value  as! Int              case "d"   : discount    = value  as! Int              case "tf"  : fare        = value  as! Int              case "ttf" : totalFare   = value  as! Int              case "hbo" : hbo         = value  as! Bool              case "pk"  : providerKey = value  as! String              case "hbf" : hbf         = value  as! [String]              default : break           }        } 

Or, if the dictionary contains NSNumbers, NSStrings, etc. and you're willing to work with the "NS" types for your internal variable, then there would be no "Dynamic" casting and you'd get a 13x performance boost for that part:

       for (key,value) in dictionary        {           switch key           {              case "rt"  : refundType  = value  as! NSString              case "r"   : score       = value  as! NSNumber              case "bf"  : baseFare    = value  as! NSNumber              case "f"   : fee         = value  as! NSNumber              case "d"   : discount    = value  as! NSNumber              case "tf"  : fare        = value  as! NSNumber              case "ttf" : totalFare   = value  as! NSNumber              case "hbo" : hbo         = value  as! NSNumber              case "pk"  : providerKey = value  as! NSString              case "hbf" : hbf         = value  as! NSArray              default : break           }        } 

Another strategy you could use is to store the dictionary inside the object without assigning the variables and make them lazy init their values:

For example:

  var defaultDict:[String:AnyObject] = [:]   lazy var score:Double = self.defaultDict["r"]  as? Double ?? 0   lazy var baseFare:Int = self.defaultDict["bf"] as? Int    ?? 0   ...    convenience init (dictionary:[String:AnyObject])    {     self.init()     defaultDict = dictionary   } 

This would only wok if the init() doesn't reference these variables for some other reasons but, it should bring the object creation time to almost nothing and defer/spread the type cast times by only performing them on actual use of the data.

Answers 3

First, use [String: Any] instead of [String: AnyObject] for dictionary in Swift.

Try using SwiftyJSON

SwiftyJSON makes it easy to deal with JSON data in Swift.

PS: Having so many if-else conditions is not a best practise.

Answers 4

You should not be checking for a value before assigning.

If all of those values are optional, you can simply assign the optional value score = dictionary["r"] as? Double and if there is no value for that key, it will simply assign it as nil.

If you want the variables to have a default value, you can nil coalesce the value score = dictionary["r"] as? Double ?? 0

This will cut the number of times you have to access the dictionary in half and also reduce the assignment of temporary variables.

Read More

When profiling NodeJS using --inspect, what does “(program)” mean

When using node --inspect to profile a NodeJS application using chrome dev tools, a large part of the runtime is spent in (program). I did not find information on what exactly this means? Is it just time spend in the C++ parts of NodeJS?

1 Answers

Answers 1

The (program) entry is Chrome itself - it's the first thing that gets executed before anything else and remains there regardless of what your Node application is doing.

There is discussion in this Chromium profiler issue on what exactly it means (and whether or not it's confusing).

Read More

Profiling Xcode tests from the command line

I've written unit tests in Xcode, using XCTest, and they are running great from Xcode. Besides the GUI environment, I also have a makefile that automates some processes from the command line. These include building, testing, cleaning, analyzing, and so on. The rationale for this is including the scripts in a build pipeline, mostly for nightly builds. With that we get builds, performance reports, memory leaks, and so on, fresh in the morning.

I'd also like to profile from the command line, using Instruments. Here's how I'm doing it:

instruments -t "Allocations" -D build/hul.trace hul.xcodeproj 

I get the following output:

Instruments Trace Error : Target failed to run: Unable to find actual binary to posix_spawn for path: hul.xcodeproj 

Is this related with the project being a static library? I'd like the tool to write the results to a file, so that I can consult in the next day, or even automate some processing, such as using pattern matching. Has anyone done something like this before using Instruments?

Edit: This seems to be doing something:

sudo instruments -v -t "Time Profiler" "build/Build/Products/Debug-iphonesimulator/hulTests.xctest/" -w "iPhone SE" 

However, then I get the following dialog:

I just don't know what the appropriate action is...

Edit: One thing was wrong. According to the docs:

 -w [hardware device, or simulator name]           The name or identifier of the hardware device or simulator to target. Should be specified first 

So the previous line changes to:

instruments  -w "iPhone SE" -v -t "Time Profiler" "build/Build/Products/Debug-iphonesimulator/hulTests.xctest/" 

Also, besides "iPhone SE" I'm using an actual device identifier, and I read somewhere that you're not supposed to use sudo. Indeed this helped! I noticed that the results for instruments -s devices and sudo instruments -s devices is actually different. So now I'm getting a different error:

Instruments : Instruments Accepted Command Instruments : Preparing To Start Trace Instruments : Starting Instrument(s) Instruments : Stopping Instrument(s) Instruments Trace Error : Target failed to run: Permission to debug com.hypelabs.test was denied. make: *** [profile] Error 253 

I googled around and this seems to indicate signing errors, but does that make sense if I'm running unit tests? That is, I'm writing a static library, so there's not even a .app file to sign.

0 Answers

Read More

How to debug time between $ionicView.beforeEnter and $ionicView.enter

I use ionic, and it takes more than 1s between $ionicView.beforeEnter and $ionicView.enter.

How can I find which part of my code is taking so much time? Batarang is not helping me much and I can't figure out an easy way of doing this...

5 Answers

Answers 1

Probably not very helpful but when I had a similar issue I could not find the culprit using Chrome debug profiler and had to comment/exclude parts of the code in my controller, which I transition to, one by one. The problem was that some third party calendar component being configured in the controller init stage was slowing the transition (view display). Once commented out everything worked fine. Since this was not my code and I did not want to mess with it I had to add a progress spinner on the transition.

Answers 2

Maybe you can use the debug tools provided with Chrome like Timeline or Profile : you start the profiling or the record of the timeline and check what happens between $ionicView.beforeEnter and $ionicView.enter. Both features have a search module so you can look for $ionicView.beforeEnter and see what is taking time until $ionicView.enter. It's probably not the easiest way but hope it will help you.

Answers 3

Have you tried monitoring the traffic in the Network tab in console? The time in ms is a good indicator of which xhr calls are running longer than expected... run a speed test.

Otherwise, if your using chrome, I would just drop some debugger statements throughout the flow of that Ionic view's state.

Answers 4

I cannot think of an easy way of doing this. But extending on what @nico_ mentioned, using the chrome javascript debug tool, you should set a breakpoint on your $ioniView.beforeEnter and a breakpoint on $ionicView.enter then step through the code between the breakpoints.

You can read more about chrome's javascript debug tools and how to set up breakpoints here: https://developer.chrome.com/devtools/docs/javascript-debugging

Answers 5

There is no code "between the breakpoints"... There are so much functions called between the 2 events...

-- Tyrius

You should try to identify functions that are taking too much time to run.

Note: I assume your app is not slowed down by downloads, you can check you downloads time in Chrome Dev Tools (If TTFB is too high, you might have a server-side slowness).

If you know which functions are called, you have two possibilities :

• When there is a few functions and not called too many times:

  function ExampleFunction() {     console.time("ExampleFunction");     // ExampleFunction code     // ...     console.timeEnd("ExampleFunction");     // output in console time between time() call and timeEnd() call } 

• If there is a lot of functions called many times :

I suggest you to use my little JS tool called MonitorJS to help you identify code blocks taking too much time to run :

function ExampleFunction() {     let mId = Monitor.Start("ExampleFunction");     // ExampleFunction code     // ...     Monitor.Stop(mId); } 

and when you what to see which function is taking too much time, call this function :

function ShowMonitorRecords() {     // get all records sorted by total_time desc     let records = Monitor.GetAll().sort((a,b)=>{ return b.total_time - a.total_time; });     // print every records     for (let record of records) {         let avg = record.total_time / record.call_count;         console.log(record.name + ": " + record.total_time.toFixed(2)              + "ms - called " + record.call_count              + " time(s) - average time : "+ avg.toFixed(2) +"ms");     } }  // will output something like : // Foo: 7234.33ms - called 2 time(s) - average time : 3617.16ms // Bar: 6104.25ms - called 3 time(s) - average time : 2034.75ms 

Once you know which function is taking too much time, you can reduce the scope of Start/Stop to identify the exact block of code slowing down your app and refactor.

Hope this helps !

Read More

VS 2015 Diagnostic Tools window is blank

I'm using Visual Studio 2015. I'm trying to use Diagnostic Tools and somehow that window is always blank. There is nothing displayed in that window. There is no error too. I tried the same with empty IDE, loading a project, and debugging the project but in all scenarios it is completely blank, just the window heading and nothing else.

I've seen other issues with Diagnostic tools such as this, and this but I can't find anybody facing issue like this.

I tried reinstalling VS and also checked by installing VS 2015 Update 2, nothing helped.

It would be great help for me if anybody got any idea why it is happening or what can I do to find the issue or fix it?

Product version: Visual Studio Enterprise Edition 2015 with Update2

OS: Windows 7 Enterprise Service pack 1

1 Answers

Answers 1

Please try to enable logs for the diagnostics tools based on instructions here. Probably, log contents can provide some hint for the reason of the issue.

Read More