Friday, June 30, 2017

AB Testing with Local Storage and Google Analytics

Leave a Comment

I'm running a sitewide AB test on my ecommerce site. After a visitor lands, I assign them a local storage key/value:

function isLocalStorageNameSupported() {   var testKey = 'test', storage = window.localStorage;   try {     storage.setItem(testKey, '1');     storage.removeItem(testKey);     return true;   } catch (error) {     return false;   } }  $(function() {    if(isLocalStorageNameSupported()){       var version = Cookies.get("version");        if (version == null) {          if (Math.random() >= 0.5){             Cookies.set("version", "A");               var version = "A"         }else{             Cookies.set("version", "B");               var version = "B"         }          ga('create', 'UA-XXXXXXXX-1', 'auto');         ga('send', {           hitType: 'event',           eventCategory: 'Pricing Experiment',           eventAction: 'New Unique Visit',           eventLabel: version,           eventValue: 0         });            }    }     }); 

After a visitor checks out, I check which version they are on and send another Google Analytics Event.

My checkout conversion events are showing up just fine. But I am only getting about 25% of the "New Unique Visit" events. Analytics is showing 12000 visits to the site but I only have 3000 of my custom events.

Which part of my code is causing this discrepancy and how can I fire an event on all visits?

1 Answers

Answers 1

This is somewhat a shot in the dark since I've ran your code and the event would trigger each time (I'm assuming your Cookies object/functions are working), but what I'm wondering is if the 12,000 number may not be correct...

Is the snippet here exactly what your site is running? Where are you sending the GA pageview? I ask because I see the event is immediately after your create function. The other potential reporting issue I see is that the event is interactive- meaning it will affect your bounce rate if you are sending a pageview too (which I'm assuming you are).

What metric specifically do you mean when you say 12,000 visits? Sessions?

Sending events before pageviews will also give you a bunch of (not set) Landing Page dimensions. That means that the event will not be associated with any landing page, and when the pageview is sent it also increments your user count... So if you're seeing more users than sessions that would indicate this is a problem.

Some things to try:

  • Send the pageview before the event.
  • Set the event to be a non-interaction event using {nonInteraction: true}.
  • Another thing I would do is only put the event within the conditional- the create and pageview send should always happen regardless of the localstorage condition.
  • Edit: You're also waiting for DOM ready to create. I'd move that to the <head> tag so the event is the only thing waiting for the document ready.

<head>      <script>  	    //Your preferred method of loading analytics.js here...  	    ga('create', 'UA-XXXXXXXX-1', 'auto');		  	    ga('send', 'pageview');      </script>  </head>

function isLocalStorageNameSupported() {  	var testKey = 'test', storage = window.localStorage;  	try {  		storage.setItem(testKey, '1');  		storage.removeItem(testKey);  		return true;  	} catch (error) {  		return false;  	}  }    $(function() {  	  	if(isLocalStorageNameSupported()){  		var version = Cookies.get("version");  		if (version == null) {  			if (Math.random() >= 0.5){  				Cookies.set("version", "A");    				var version = "A"  			}else{  				Cookies.set("version", "B");    				var version = "B"  			}  			  			ga('send', 'event', 'Pricing Experiment', 'New Unique Visit', version, 0, {nonInteraction: true});			  		}   	}     });

See if that gets your metrics closer together, and then consider this alternative way of tracking the data:

Perhaps an event here isn't the best solution altogether. What you're doing sounds like an excellent candidate for using a custom dimension scoped to the user.

I'd recommend creating a custom dimension called "Pricing Experiment" with a scope of User (because with GA Experiments, the user will always see the same variation upon successive sessions) and then after your create function, replace the event with this: ga('set', 'dimension1', version); (be sure to replace dimension1 with your actual number.

Finally, send a pageview after setting the dimension (custom dimensions use hit types to be transported to GA).

<head>  	<script>  		//Your preferred method of loading analytics.js here...  		ga('create', 'UA-XXXXXXXX-1', 'auto');  		  		if(isLocalStorageNameSupported()){  			var version = Cookies.get("version");  			if (version == null) {  				if (Math.random() >= 0.5){  					Cookies.set("version", "A");    					var version = "A"  				}else{  					Cookies.set("version", "B");    					var version = "B"  				}  				  				ga('set', 'dimension1', version);			  			}   		}   		  		ga('send', 'pageview');  		  		function isLocalStorageNameSupported() {  			var testKey = 'test', storage = window.localStorage;  			try {  				storage.setItem(testKey, '1');  				storage.removeItem(testKey);  				return true;  			} catch (error) {  				return false;  			}  		}  		  	</script>  </head>

Now when you view any other report, you can apply a Secondary Dimension of "Pricing Experiment" and you'll be able to show which version they saw.

This way you'll be able to get the contextual data without messing with hit types that alter reporting metrics.

Again, I apologize if this isn't perfectly illuminating your issue- I'm taking the snippet you provided literally and filling in any blanks as best I can.

Read More

How can I quickly enumerate directories on Win32?

Leave a Comment

I'm trying to speedup directory enumeration in C++, where I'm recursing into subdirectories. I currently have an app which spends 95% of it's time in FindFirst/FindNextFile APIs, and it takes several minutes to enumerate all the files on a given volume. I know it's possible to do this faster because there is an app that does: Everything. It enumerates my entire drive in seconds.

How might I accomplish something like this?

6 Answers

Answers 1

I realize this is an old post, but there is a project on source forge that does exactly what you are asking and the source code is available.

You can find the project here: NTFS-Search

Answers 2

"Everything" accesses directory information at a lower level than the Win32 FindFirst/FindNext APIs.

I believe it reads and interprets the NTFS MFT structures directly, and that this is one of the main reasons for its performance. It's also why it requires admin privileges and why "Everything" only indexes local or removable NTFS volumes (not network drives, for example).

A couple other utilities that do the similar things are:

A little reverse engineering with a debugger on these tools might give you some insight on the techniques they use.

Answers 3

"Everything" builds an index in the background, so queries are against the index not the file system itself.

There are a few improvements to be made - at least over the straight-forward algorrithm:

First, breadth search over depth search. That is, enumerate and process all files in a single folder before recursing into the sub folders you found. This improves locality - usually a lot.

On Windows 7 / W2K8R2, you can use FindFirstFileEx with FindExInfoBasic, the main speedup being omitting the short file name on NTFS file systems where this is enabled.

Separate threads help if you enumerate different physical disks (not just drives). For the same disk it only helps if it's an SSD ("zero seek time"), or you spend significant time processing a file name (compared to the time spent on disk access).


[edit] Wikipedia actually has some comments - Basically, they are skipping the file system abstraction layer, and access NTFS directly. This way, they can batch calls and skip expensive services of the file system - such as checking ACL's.

A good starting point would be the NTFS Technical Reference on MSDN.

Answers 4

If you are doing this on NTFS, here's a lib for low level access: NTFSLib.

You can enumerate through all file records in $MFT, each representing a real file on disk. You can get all file attributes from the record, including $DATA.

This may be the fastest way to enumerate all files/directories on NTFS volumes, 200k~300k files per minute as I tested.

Answers 5

Don't recurse immediately, save a list of directories you find and dive into them when finished. You want to do linear access to each directory, to take advantage of locality of reference and any caching the OS is doing.

Answers 6

If you're already doing the best you can to get the maximum speed from the API, the next step is to do low-level disk accesses and bypass Windows altogether. You might get some guidance from the NTFS drivers for Linux, or perhaps you can use one directly.

Read More

Java Hibernate org.hibernate.exception.SQLGrammarException: could not extract ResultSet on createSQLQuery

Leave a Comment

I have this method.

private final void updateAllTableFields(final Class clazz){     final String tableName = ((Table)clazz.getAnnotation(Table.class)).name();     final String sqlQuery = new StringBuilder("SET @ids = NULL; ")             .append("UPDATE ")             .append(tableName)             .append(' ')             .append("set activeRecord=:activeRecord ")             .append("where activeRecord=true and updateable=true ")             .append("and (SELECT @ids \\:= CONCAT_WS(',', id, @ids)); ")             .append("select @ids;")             .toString();     final Query query = session.createSQLQuery(sqlQuery)             .setParameter("activeRecord",Boolean.FALSE);     final Object idsList=query.uniqueResult();     System.out.println("idsList = " + idsList); }         

I want to do a update and also return the affected Ids this works Perfect using a rawSQL returns the id in a string fashion but i couldn't make it work using Hibernate any tip!!!

Thanks in advance and best regards.

UPDATE

I need to do a update and return the affected id!! I dont want to make a simple UPDATE.

you can check it out the original question here pal: https://stackoverflow.com/questions/44604763/java-hibernate-tips-about-update-all-table-fields-performance

UPDATE The error is

at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:80) at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:126) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:112) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:89) at org.hibernate.loader.Loader.getResultSet(Loader.java:2065) at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1862) at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1838) at org.hibernate.loader.Loader.doQuery(Loader.java:909) at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:354) at org.hibernate.loader.Loader.doList(Loader.java:2553) at org.hibernate.loader.Loader.doList(Loader.java:2539) at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2369) at org.hibernate.loader.Loader.list(Loader.java:2364) at org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:353) at org.hibernate.internal.SessionImpl.listCustomQuery(SessionImpl.java:1873) at org.hibernate.internal.AbstractSessionImpl.list(AbstractSessionImpl.java:311) at org.hibernate.internal.SQLQueryImpl.list(SQLQueryImpl.java:141) at org.hibernate.internal.AbstractQueryImpl.uniqueResult(AbstractQueryImpl.java:966) at company.nuevemil.code.finalizarEntornoDePrueba(Test.java:56) at company.nuevemil.code.main(Test.java:27)   Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'UPDATE student set activeRecord=false,uid=1 where activeRecord=true at line 1 

4 Answers

Answers 1

you have to use HQL Query for bulk update. you are going write way only thing is that, you have to create HQL query for example       Your Query Might be like this:-     final String tableName = ((Table)clazz.getAnnotation(Table.class)).name();         final String sqlQuery = new StringBuilder("SET @ids = NULL; ")                 .append("UPDATE ")                 .append(tableName)                 .append(' ')                 .append("set activeRecord=:activeRecord ")                 .append("where activeRecord=true and updateable=true ")                 .append("and (SELECT @ids \\:= CONCAT_WS(',', id, @ids)); ")                 .append("select @ids;")                 .toString();         final Query query = session.createQuery(sqlQuery)                 .setParameter("activeRecord",Boolean.FALSE);         final Object idsList=query.executeUpdate();      Example Query:     final String tableName = ((Table)clazz.getAnnotation(Table.class)).name();        Query qry = session.createQuery("update "+tableName+" p set p.proName=?     where p.productId=111");                 qry.setParameter(0,"updated..");                 int res = qry.executeUpdate(); 

Answers 2

There is no "affected id" in an UPDATE statement.

UPDATE student     set activeRecord=false,uid=1     where activeRecord=true 

may modify 0 rows, 1 rows, or many rows.

What is the PRIMARY KEY of student? Let's say it is studentId. To retrieve all (if any) of the studentId values, you neecd the Hibernate equivalent of this pseudo-code:

START TRANSACTION; @ids = SELECT studentId            FROM student            WHERE activeRecord=true  -- and updateable=true ??            FOR UPDATE; UPDATE student     SET activeRecord=false,         uid=1     WHERE activeRecord=true  -- and updateable=true ??     ; COMMIT; 

More

That code could be bundled up in a Stored Procedure, thereby allowing it to be CALLed as if a single statement. (I do not know how to make it work with Hibernate.)

Answers 3

I suppose, you won't be able to make it in Hibernate fashion.

Hibernate is independent from a database. But the part of the query that initializes a variable (I mean set @ids = null;) is not portable across all the relational databases so I wouldn't expect it to be in Hibernate API somewhere.

Answers 4

I would sugest extracting records to be updated as list of entity, then you can iterate to set values, persist and still return afected ids at the end of your method

Read More

Thursday, June 29, 2017

.HTACCESS Redirection Issues

Leave a Comment

Namely, I have this .htaccess command...

RewriteCond %{REQUEST_FILENAME} -f [OR] RewriteCond %{REQUEST_FILENAME} -d RewriteRule ^ - [L] RewriteRule ^([^\x00-\x7F]+).*$ ?open=encyclopedia&letter=$1&term=$0 [R,B,L,QSA] RewriteRule ^([A-Z](?:[^\x00-\x7F]+|[A-Z])?).*$ ?open=encyclopedia&letter=$1&term=$0 [R,B,L,QSA] 

...And I have two issues with it now:
Issue 1 - It loads shorthand of all of the letters except A, S and O. It displays blank white page instead of the actual page.
Issue 2 - When I enter http://example.com/Šandi (Astronomy), instead of redirecting me normally to http://example.com/?open=encyclopedia&letter=Š&term=Šandi+(Astronomy), the URL is http://example.com/?open=encyclopedia&letter=%25c5%25a0&term=%25c5%25a0andi+%2528Astronomy%2529

In other words:
• When I remove the [R] flag from either of the Rules, the shorthand URL (id est - example.com/A-O-S) is as described in Issue 1.
• When I add the [R] flag, it redirects inappropriately, as described in Issue 2, and it does not display the page.

Note: These problems include English and non-English letters (Š/Đ/Č/Ć/Ž).

As @anubhava suggested:
<?php print_r($_GET); ?> with [R] flag:
English: Array ( [open] => encyclopedia [letter] => V [term] => Vodolija (Astrologija) )
URL: ?open=encyclopedia&letterV&term=Vodolija (Astrologija)
Non-English: Array ( [open] => encyclopedia [letter] => ? [term] => Škorpija (Astrologija) )
URL: ?open=encyclopedia&letter=%c5&term=%c5%a0korpija%20(Astrologija)

<?php print_r($_GET); ?> without [R] flag:
English: Array ( [open] => encyclopedia [letter] => V [term] => Vodolija (Astrologija) )
URL: /Vodolija (Astrologija)
Non-English: Array ( [open] => encyclopedia [letter] => Š [term] => Škorpija (Astrologija) )
URL: /Škorpija (Astrologija)

As for the Index pages:
A - Array ( [open] => encyclopedia [letter] => A [term] => A ): It doesn't display the page, it shows blank white page.
O - Array ( [open] => encyclopedia [letter] => O [term] => O ): It doesn't display the page, it shows blank white page.
S - Array ( [open] => encyclopedia [letter] => S [term] => S ): It doesn't display the page, it shows blank white page.
As for the all the other Index pages, URL is Array ( [open] => encyclopedia [letter] => AnyLetter [term] => SameAnyLetter): It displays the page correctly... ...I do not see logic in it.

This is the new code:

RewriteCond %{REQUEST_FILENAME} -f [OR] RewriteCond %{REQUEST_FILENAME} -d RewriteRule ^ - [L] #1 RewriteRule ([A-Z](?:[^\x00-\xFF].|\w).*) ?open=encyclopedia&letter=$2 [R,B,NE,L,QSA] # it loads DZH, but not the rest #2 RewriteRule (([\x00-\xFF].|\w).*) ?open=encyclopedia&letter=$2 [R,B,NE,L,QSA] # it loads A-O-S and it loads DJ-ZH-LJ-NJ-SH RewriteRule ^([A-Z](?:[^\x00-\x7F]+|[A-Z])?).*$ ?open=encyclopedia&letter=$1&term=$0 [B,NE,L,QSA] 

When I delete the line #1, the line #2 is working fine. When I delete the line #2, the line #1 is working okay. But they won't work together. The line #2 displays all the letters except for the DŽ, and redirects properly. The line #1 displays only the letter DŽ properly, won't redirect and won't load the rest. With the line #2 enabled, the URL with /DŽ redirects to /?open=encyclopedia&letter=D%c5

UPDATE #6:

With the following code...

RewriteCond %{REQUEST_FILENAME} -f [OR] RewriteCond %{REQUEST_FILENAME} -d RewriteRule ^ - [L] RewriteRule ((LJ).*) ?open=encyclopedia&letter=$2&term=$1 [R,NE,L,QSA] RewriteRule ((NJ).*) ?open=encyclopedia&letter=$2&term=$1 [R,NE,L,QSA] RewriteRule ((D\xC5\xBD).*) ?open=encyclopedia&letter=$2&term=$1 [R,NE,L,QSA] RewriteRule (([\x00-\xFF].|\w).*) ?open=encyclopedia&letter=$2&term=$1 [R,NE,L,QSA] 

...This happens:
• INDEX Pages: As for the „double-letter“ letters, it opens every letter correctly. As of the single letters, it loads every letter except A, O and S.
• TERM Pages: Considering it won't open A, O and S, therefore it won't open their Terms pages.
/Lav (Astrologija) doesn't redirect to /?open=encyclopedia&letter=L&term=Lav (Astrologija), but instead redirects to /?open=encyclopedia&letter=L&term=Lav+%2528Astrologija%2529 thus won't open the proper page.
• It opens properly and it redirects /Škorpija (Astrologija) properly to /?open=encyclopedia&letter=Š&term=Škorpija (Astrologija)
• It opens properly and it redirects /Đavo properly to /?open=encyclopedia&letter=Đ&term=Đavo
• Although it opens /U properly, /Unuk Al Haj redirects to /?open=encyclopedia&letter=Un&term=Unuk%20Al%20Haj, not opening the right page.
• Although it opens /U properly, /Ugaoni Razmak redirects to /?open=encyclopedia&letter=Ug&term=Unuk%20Al%20Haj, not opening the right page.
• Although it opens /R properly, /Ribe (Astrologija) redirects to /?open=encyclopedia&letter=Ri&term=Ribe (Astrologija), not opening the right page.
• Although it opens /V properly, /Vaga (Astrologija) redirects to /?open=encyclopedia&letter=Va&term=Vaga (Astrologija), not opening the right page.
• Although it opens /U properly, /Umerenost redirects to /?open=encyclopedia&letter=Um&term=Umerenost, not opening the right page.
• Although it opens /K properly, /Kuće (Astrologija) redirects to /?open=encyclopedia&letter=Ku&term=Kuće (Astrologija), not opening the right page.

2 Answers

Answers 1

It seems to me that you should be able to use:

RewriteCond %{REQUEST_FILENAME} -f [OR] RewriteCond %{REQUEST_FILENAME} -d RewriteRule ^ - [L]  RewriteRule ^(.)[^/]*$ ?open=encyclopedia&letter=$1&term=$0 [L,QSA] 

Answers 2

Use NE (no escape) flag:

# here are all 2 letters which we count as 1  RewriteRule ((D\xC5\xBD).*) ?open=encyclopedia&letter=$2&term=$1 [R,NE,L,QSA] # C5 BD is utf8 encoding of Ž RewriteRule ((LJ).*) ?open=encyclopedia&letter=$2&term=$1 [R,NE,L,QSA] RewriteRule ((NJ).*) ?open=encyclopedia&letter=$2&term=$1 [R,NE,L,QSA]  # one letter RewriteRule (([\x80-\xFF].|\w).*) ?open=encyclopedia&letter=$2&term=$1 [R,NE,L,QSA] 
Read More

Android microphone constantly gives 32639 or -32640 on newer devices

Leave a Comment

I've implemented code similar to this. I have a noise alert go off in the Log, but it always gives 32639 or -32640 regardless of what kind of noise is going on outside.

short[] buffer = new short[minSize]; boolean thresholdMet = false; int threshold = sliderThreshold.getProgress();  ar.read(buffer, 0, minSize);  //Iterate through each chunk of amplitude data //Check if amplitude is greater than threshold for (short s : buffer) {     if (Math.abs(s) > threshold) {         thresholdMet = true;         Log.w("NoiseThreshold", String.valueOf(s));         break;     } } 

I've tested it on three phones (none of which are rooted):

  • Samsung Galaxy S3(API 19)
  • HTC One M9(API 23)
  • Samsung Galaxy S7(API 24)

It works on the S3, but not the others. I've tried using Sensor Sense on the HTC and it doesn't work for the mic sensor. However, it used to work, and now seems to detect one sample every five seconds or so in the graph view.

Oddly enough, the microphone still works fine for phone calls and video recording on the malfunctioning phones.

1 Answers

Answers 1

You said it works on S3, which is API 19 and doesn't on those with API>=23. So, it's possible that you have a problem with runtime permissions introduced in API 23.

New behaviour (for "old apps" which use static permission model) is to return dummy data if runtime permission is not granted.

Check out this answer: Request permission for microphone on Android M

Read More

Firebase multi-tenancy with play framework depends on header value of HTTP request

Leave a Comment

I have a play framework project that provides APIs that shared between multiple front-ends, currently, I'm working on single front-end but I want to create a multi-tenant backend, each front-end got its own Firebase account.

My problem that I have to consider which firebase project to access depends on the request header value, that came with different values depends on the front end.

What I have now: FirebaseAppProvider.java:

public class FirebaseAppProvider implements Provider<FirebaseApp> {       private final Logger.ALogger logger;     private final Environment environment;     private final Configuration configuration;      @Inject     public FirebaseAppProvider(Environment environment, Configuration configuration) {         this.logger = Logger.of(this.getClass());         this.environment = environment;         this.configuration = configuration;     }      @Singleton     @Override     public FirebaseApp get() {         HashMap<String, String> firebaseProjects = (HashMap<String, String>) configuration.getObject("firebase");         firebaseProjects.forEach((websiteId, projectId) -> {             FileInputStream serviceAccount = null;             try {                 serviceAccount = new FileInputStream(environment.classLoader().getResource(String.format("firebase/%s.json", projectId)).getPath());             } catch (FileNotFoundException e) {                 e.printStackTrace();                 return;             }              FirebaseOptions options = new FirebaseOptions.Builder().setCredential(FirebaseCredentials.fromCertificate(serviceAccount))                     .setDatabaseUrl(String.format("https://%s.firebaseio.com/", projectId))                     .build();               FirebaseApp firebaseApp = FirebaseApp.initializeApp(options, projectId);              logger.info("FirebaseApp initialized");         });          return FirebaseApp.getInstance();     } } 

Also for Database: FirebaseDatabaseProvider.java

public class FirebaseDatabaseProvider implements Provider<FirebaseDatabase> {      private final FirebaseApp firebaseApp;     public static List<TaxItem> TAXES = new ArrayList<>();      @Inject     public FirebaseDatabaseProvider(FirebaseApp firebaseApp) {         this.firebaseApp = firebaseApp;         fetchTaxes();     }      @Singleton     @Override     public FirebaseDatabase get() {         return FirebaseDatabase.getInstance(firebaseApp);     }      @Singleton     public DatabaseReference getUserDataReference() {         return this.get().getReference("/usersData");     }      @Singleton     public DatabaseReference getTaxesConfigurationReference() {         return this.get().getReference("/appData/taxConfiguration");     }     private void fetchTaxes() {         DatabaseReference bundlesRef = getTaxesConfigurationReference().child("taxes");         bundlesRef.addValueEventListener(new ValueEventListener() {             @Override             public void onDataChange(DataSnapshot dataSnapshot) {                 TAXES.clear();                 dataSnapshot.getChildren().forEach(tax -> TAXES.add(tax.getValue(TaxItem.class)));                 Logger.info(String.format("==> %d taxes records loaded", TAXES.size()));             }              @Override             public void onCancelled(DatabaseError databaseError) {                 Logger.warn("The read failed: " + databaseError.getCode());             }         });     } } 

So I bind them as well from Module.java:

public class Module extends AbstractModule {      @Override     public void configure() {        bind(FirebaseApp.class).toProvider(FirebaseAppProvider.class).asEagerSingleton();         bind(FirebaseAuth.class).toProvider(FirebaseAuthProvider.class).asEagerSingleton();         bind(FirebaseDatabase.class).toProvider(FirebaseDatabaseProvider.class).asEagerSingleton();     }  } 

my ActionCreator:

public class ActionCreator implements play.http.ActionCreator {      @Inject     public ActionCreator() {     }      @Override     public Action createAction(Http.Request request, Method actionMethod) {         switchTenancyId(request);         return new Action.Simple() {             @Override             public CompletionStage<Result> call(Http.Context ctx) {                 return delegate.call(ctx);             }         };     }      private void switchTenancyId(Http.RequestHeader request) {         // DO something here     }      private Optional<String> getTenancyId(Http.RequestHeader request) {         String websiteId = request.getHeader("Website-ID");         System.out.println(websiteId);         return null;     } } 

What I want is when I use Database service, or auth service, I read the website id and decide which firebase project to access, I really tried the solution like this answer here: Multi tenancy with Guice Custom Scopes and Jersey

Please note I'm willing to use differents projects, not the same firebase project for each front-end.

But kinda lost, especially the request can be only accessed from controller or ActionCreator, so what I got from the question above is load providers by key into ThreadLocal and switch them for each request depends on the annotation, but I was unable to do this because of the lack of knowledge.


The minimized version of my project can be found here: https://github.com/almothafar/play-with-multi-tenant-firebase

Also, I uploaded taxes-data-export.json file to import inside firebase project for a test.

2 Answers

Answers 1

I believe Custom Scopes for this is overkill. I would recommend doing the Request-Scoped seeding from Guice's own wiki. In your case that would be something like

public class TenancyFilter implements Filter {     @Override     public void doFilter(ServletRequest request,  ServletResponse response, FilterChain chain) throws IOException, ServletException {         HttpServletRequest httpRequest = (HttpServletRequest) request;         String tenancyId = httpRequest.getHeader("YOUR-TENANCY-ID-HEADER-NAME");         httpRequest.setAttribute(                 Key.get(String.class, Names.named("tenancyId")).toString(),                 userId         );         chain.doFilter(request, response);     }      @Override     public void init(FilterConfig filterConfig) throws ServletException { }      @Override     public void destroy() { } }; 

It has to be bound in a ServletModule

public class YourModule extends ServletModule {     @Override     protected void configureServlets() {         filter("/*").through(TenancyFilter.class);     }      @Provides     @RequestScoped     @Named("tenancyId")     String provideTenancyId() {         throw new IllegalStateException("user id must be manually seeded");     } } 

Then anywhere you need to get the Tenancy ID you just inject

public class SomeClass {     private final Provider<String> tenancyIdProvider;      @Inject     SomeClass(@Named("tenancyId") Provider<String> tenancyIdProvider) {         this.tenancyIdProvider = tenancyIdProvider;     }      // Methods in request call tenancyIdProvider.get() to get and take action based on Tenancy ID. } 

Answers 2

Right, so I know Play a lot better than FireBase, but it seems to me you want to extract a tenancy ID from the request prior to feeding this into your FrieBase backend? Context when writing Java in play is Thread local, but even when doing things async you can make sure the Http.context info goes along for the ride by injecting the execution context. I would not do this via the action creator, unless you want to intercept which action is called. (Though I have a hackish solution for that as well.)

So, after a comment I'll try to elucidate here, your incoming request will be routed to a controller, like below (let me know if you need clearing up on routing etc):

Below is a solution for caching a retrieved FireBaseApp based on a "Website-ID" retrieved from the request, though I would likely put the tenancyId in the session.

import javax.inject.Inject; import java.util.concurrent.CompletionStage;  public class MyController extends Controller {     private HttpExecutionContext ec; //This is the execution-context.     private FirebaseAppProvider appProvider;     private CacheApi cache;     @Inject     public MyController(HttpExecutionContext ec, FireBaseAppProvider provider,CacheApi cache) {         this.ec = ec;         this.appProvider = provider;         this.cache = cache;      }     /**     *Retrieves a website-id from request and attempts to retrieve      *FireBaseApp object from Cache.     *If not found a new FireBaseApp will be constructed by      *FireBaseAppProvider and cached.     **/     private FireBaseApp getFireBaseApp(){      String tenancyId = request.getHeader("Website-ID);      FireBaseApp app = (FireBaseApp)cache.get(tenancyId);      if(app==null){        app=appProvider.get();        cache.put(tenancyId,app);        }      return app;     }         public CompletionStage<Result> index() {         return CompletableFuture.supplyAsync(() -> {            FireBaseApp app = getFireBaseApp();            //Do things with app.         }, ec.current()); //Used here.     } } 

Now in FireBaseAppProvider you can access the header via play.mvc.Controller, the only thing you need is to provide the HttpExecutionContext via ec.current. So (once again, I'm avoiding anything FireBase specific), in FireBaseProvider:

import play.mvc.Controller; public class FireBaseAppProvider {   public  String getWebsiteKey(){         String website = Controller.request().getHeader("Website-ID");         //If you need to handle a missing header etc, do it here.         return website;     }   public FireBaseApp get(){      String tenancyId = getWebsiteKey();      //Code to do actual construction here.   } } 

Let me know if this is close to what you're asking and I'll clean it up for you.

Also, if you want to store token validations etc, it's best to put them in the "session" of the return request, this is signed by Play Framework and allows storing data over requests. For larger data you can cache this using the session-id as part of the key.

Read More

Module using another function of the same module

Leave a Comment

I have something like this:

  • MyModule
    • index.js
    • myFunction1.js
    • myFunction2.js

In index.js file i'm exporting all modules function (myFunction1 and myFunction2). But, in myFunction2 I use myFunction1.

If I import the index (all the module) and call it like MyModule.myFunction1 inside myFunction2, I get an error (function does not exists).

If I import myFunction1.js and call it like myFunction1(), I can't make an Stub of it when I'm going to test it.

Any aproach to do something like this?

6 Answers

Answers 1

////////////////////// // File: ./index.js // ////////////////////// const myFunction1 = require('./myFunction1.js'); const myFunction2 = require('./myFunction2.js');  module.exports = {     myFunction1,     myFunction2 };  //////////////////////////// // File: ./myFunction1.js // //////////////////////////// module.exports = function1(x){     return x; };  //////////////////////////// // File: ./myFunction2.js // //////////////////////////// const myFunction1 = require('./myFunction1.js'); // or dropping the `.js` require('./myFunction1') module.exports = function2(x){     return myFunction1(x); }; 

Answers 2

index.js

let fnk1 = (a) => {   return a + ' 1 ' } let fnk2 = (a) => {   return fnk1(a) } module.exports = {  fnk1,  fnk2 } 

test.js

let Mymodule = require('./index') console.log(Mymodule.fnk2('argValue')); 

Answers 3

This should do the trick. It would be better if they were in the same file.

module.exports.myFunction1 = function() { ... }  module.exports.myFunction2 = function() {    module.exports.myFunction1() } 

If you need them to be in separate files, you can require function1 inside of the function2 file. The thing is, each file is its own module. From the Node.js docs:

Node.js has a simple module loading system. In Node.js, files and modules are in one-to-one correspondence (each file is treated as a separate module).

Answers 4

Diego's answer works fine, except that OP seems to want to use index.js in myFunction2.js. Use the same code as Diego's for index.js

for index.js (same as Diego's)

const myFunction1 = require('./myFunction1.js'); const myFunction2 = require('./myFunction2.js');  module.exports = {     myFunction1,     myFunction2 }; 

for myFunction1.js (small fix)

module.exports = function function1(x){     return x; }; 

for myFunction2.js (changed to include index.js):

const MyModule = require('./index.js');  module.exports = function function2(x){     return MyModule.myFunction1(x); }; 

and a test.js :

const myFunction2 = require('./myFunction2.js');  console.log(myFunction2(20)) 

Answers 5

const myFunction1 = require('./myFunction1.js');  const myFunction2 = require('./myFunction2.js');  module.exports = {      myFunction1:myFunction1,      myFunction2:function(x){        return myFunction2(x,myFunction1);      }  }  ////////////////////////////  // File: ./myFunction1.js //  ////////////////////////////  module.exports = function1(x){      return x;  };    ////////////////////////////  // File: ./myFunction2.js //  ////////////////////////////    module.exports = function2(x,myFunction1){      return myFunction1(x);  };

Answers 6

You problem is not clear because you did not post your code. Since, here is a working solution for your problem:

MyModule.js:

function myFunction1() {   console.log("- I am myFunction1") }  function myFunction2() {   console.log("- I am myFunction2")   myFunction1() }  module.exports = {   myFunction1,   myFunction2 } 

index.js

const MyModule = require('./MyModule')  console.log("Call myFunction1") MyModule.myFunction1() console.log("Call myFunction1") MyModule.myFunction2() 

If you execute node index.js, you get the following output:

Call myFunction1 - I am myFunction1 Call myFunction2 - I am myFunction2 - I am myFunction1 
Read More

How to prevent checkboxes and dropdowns in a div to be scaled in safari

Leave a Comment

I have a div which contains some elements. I want the div to be scaled when it is hovered upon. It works fine in chrome but something weird happens in safari. The checkboxes and dropdowns are also getting scaled. How can I fix it in safari?

.box:hover {    transform: scale(1.03);    -ms-transform: scale(1.03);    -webkit-transform: scale(1.03);    -webkit-transition: all .01s ease-in-out;    transition: all .01s ease-in-out;  }    div {    padding-left: 30px;    margin: 10px;  }    .box {    border: 1px solid black;  }
<div class="box">    <div>      <input type="checkbox" name="opt1" id="option1" /> hello    </div>    <div>      <select>        <option>apple</option>        <option>orange</option>      </select>    </div>    <div>      <input type="text" placeholder="enter something" />    </div>  </div>

5 Answers

Answers 1

This isn't browser specific issue, the transform scale property is working the way it should work, it will scale each and every nested element(s) to 1.03 within .box element.

The only way you have is to use inverse scaling to child elements 1 / 1.03 = 0.97

.box:hover {    transform: scale(1.03);    -ms-transform: scale(1.03);    -webkit-transform: scale(1.03);    -webkit-transition: all .01s ease-in-out;    transition: all .01s ease-in-out;  }    div {    padding-left: 30px;    margin: 10px;  }    .box:hover *{    transform: scale(0.97);    -ms-transform: scale(0.97);    -webkit-transform: scale(0.97);    -webkit-transition: all .01s ease-in-out;    transition: all .01s ease-in-out;  }    .box {    border: 1px solid black;  }
<div class="box">    <div>      <input type="checkbox" name="opt1" id="option1" /> hello    </div>    <div>    <select>      <option>apple</option>      <option>orange</option>    </select>    </div>    <div>      <input type="text" placeholder="enter something" />    </div>  </div>

Source

Answers 2

Another solution would be to move the scale effect to a pseudo element which behaves exactly the same as your main div .box. This pseudo element will adapt the height and width of this div.

The advantage with this solution would be that the content of your div .box wouldn't get affected by the scale anymore since the scale doesn't happen on this div. In addition, you don't have to change anything in your HTML structure.

div {    padding-left: 30px;    margin: 10px;  }    .box {    position: relative;    padding: 10px;  }    .box:after {    content: '';    position: absolute;    top: 50%;    left: 50%;    width: 100%;    height: 100%;    transform: translate(-50%, -50%) scale(1);    border: 1px solid black;    transition: 0.3s;    z-index: -1;  }    .box:hover:after {    transform: translate(-50%, -50%) scale(1.03);  }
<div class="box">    <div>      <input type="checkbox" name="opt1" id="option1" /> hello    </div>    <div>      <select>        <option>apple</option>        <option>orange</option>      </select>    </div>    <div>      <input type="text" placeholder="enter something" />    </div>  </div>

Answers 3

You can add this to your css on the checkboxes:

-webkit-transform:scale(3, 3); 

Answers 4

div {    padding-left: 30px;    margin: 10px;  }    .box {    position: relative;    padding: 10px;  }    .box:after {    content: '';    position: absolute;    top: 0;    left: 0;    width: 100%;    height: 100%;    transform:scale(1);    border: 1px solid black;    transition: 0.3s;  }    .box:hover:after {    transform:scale(1.03);  }
<div class="box">    <div>      <input type="checkbox" name="opt1" id="option1" /> hello    </div>    <div>      <select>        <option>apple</option>        <option>orange</option>      </select>    </div>    <div>      <input type="text" placeholder="enter something" />    </div>  </div>

I have removed transform property from @SvenL answer. without that too its working fine in safari.

Answers 5

Yeah with just HTML,CSS alone you cannot style differently for Safari, Chrome and Opera since all of the three uses the same flag -webkit- for styling using css

so the possible solutions is to Detect the browser using javascript and add a class to your body to indicate the browser

Eg. if you wanna style safari you will be styling

.safari .box:hover{

     /* style as you need */ 

}

the test code i had run is attached below.

<html> <head>     <title>TVEK Test App for Abhishek Pandey</title> </head> <body>     <script   src="https://code.jquery.com/jquery-3.2.1.min.js"   integrity="sha256-hwg4gsxgFZhOsEEamdOYGBf13FyQuiTwlAQgxVSNgt4="   crossorigin="anonymous"></script>     <script type="text/javascript">         $(function () {             var userAgent = navigator.userAgent.toLowerCase();              if (userAgent .indexOf('safari')!=-1){                  if(userAgent .indexOf('chrome')  > -1){                     //browser is chrome                     alert('chrome');                 }else if((userAgent .indexOf('opera')  > -1)||(userAgent .indexOf('opr')  > -1)){                     //browser is opera                      alert('opera');                 }else{                     //browser is safari, add css                     alert('safari');                     $("body").addClass("safari");                 }             }         });     </script>     <style type="text/css">     .safari .box{         /* add your required style here */       margin:20px;     }      .box:hover {       transform: scale(1.03);       -ms-transform: scale(1.03);       -webkit-transform: scale(1.03);       -webkit-transition: all .01s ease-in-out;       transition: all .01s ease-in-out;     }      div {       padding-left: 30px;       margin: 10px;     }      .box {       border: 1px solid black;     }     </style>      <div class="box">         <div>             <input type="checkbox" name="opt1" id="option1" /> hello         </div>         <div>             <select>                 <option>apple</option>                 <option>orange</option>             </select>         </div>         <div>             <input type="text" placeholder="enter something" />         </div>     </div> </body> </html> 

Please credit the bounty if my solution is helpful

you can remove the alert if required. I just added the alert to show you the Proof-of-Evidence.

Read More

Using the Django ORM, How can you create a unique hash for all possible combinations

Leave a Comment

I want to maintain a Django model with a unique id for every combination of choices within the model. I would then like to be able to update the model with a new field and not have the previous unique id's change. The id's can be a hash or integer or anything. What's the best way to achieve this?

class MyModel(models.Model):     WINDOW_MIN = 5     WINDOW_MAX = 7     WINDOW_CHOICES = [(i,i) for i in range(WINDOW_MIN - 1, WINDOW_MAX - 1)]     window = models.PositiveIntegerField('Window', editable=True, default=WINDOW_MIN, choices=WINDOW_CHOICES)     is_live = models.BooleanField('Live', editable=True, default=False)     unique_id = .... 

Given the above example there will be 3 * 2 == 6 unique id's. If I add another editable boolean field, I don't want to change the previous unique id's but I want the new unique id's to be generated for the new boolean field.

The thought process behind this is the parameters in MyModel define the inputs to a function who's results are stored in another Django model MyResultModel by unique_id and the name of the model. The reasoning behind this is there are multiple variants of MyModel each with it's own set unique combination's that get updated regularly but the result set in MyResultModel is the same across MyModel1 to MyModelN. Ideally I would like the unique_id's to be autogenerated. In other words the key for the result set stored in MyResultModel is the model_name (MyModel) and a unique_id. I want to sanely manage this many (MyModel1,...MyModelN) to one (MyResultModel) relationship.

class MyResultModel(models.Model):     unique_id = ...     model_name = models.CharField(max_length=200, default='', blank=False) # Points to a Django Model ex MyModel     result1 = ...     result2 = ... 

2 Answers

Answers 1

A common approach, given that all your options are boolean, categorical or small numbers, you could just pack them into a bigger bit field (https://en.wikipedia.org/wiki/Bit_field) and whenever you add a new option, make sure to push it to the most-significant part of your bit field and avoid having a value of 0 (that is, simple add 1 to whatever). Not only would every single unique_id represent a different configuration, you could even do without the model and just use bit operations to extract all your values from the bit field.

Answers 2

I have 2 thing: First, if you just want to create unique hash for all the combinations

# Just set it all together like uniq= cb1+cb2+cb3... (a string key is ok =) ) # If it is very very long string ~> we think about using hash to short it (and have same size for all value). 

And next:

For your question above:

i can't understand why do you make the model into a complicate thing (like a mess) but still have to solve it:

As i read your ques: you have many variants of Models ~> want to gather it into 1 model res?

So: it is better to set FK

map = models.ForeignKey(Model_name, null=True) # but if you have too many model, it can't be help... 

So i recomment:

~ create just 1 model, because if you have too many model, you can't even call it to get data (As you did) but when you add more FIELD, unique id ~> should change.

One good way to have special unique_id:

Use the first solution i said.

Or:

Create NEW table just to store YOUR COMBO:

New table will have ALL field and you will SET value for each (or you can write script to create it). ~> You have id for each combo ~> it same as unique id

You can create ALL THE POSSIBLE combo or just check and add when the NEW combo show up

Read More

Improving performance of hive jdbc

Leave a Comment

Does aynyone know how to increase performance for HIVE JDBC connection.

Detailed problem:

When I query hive from Hive CLI, I get a response within 7 sec but from HIVE JDBC connection I get a response after 14 sec. I was wondering if there is any way (configuration changes) with which I can improve performance for query through JDBC connection.

Thanks in advance.

2 Answers

Answers 1

Can you please try the below options.

  1. If your query has joins then try setting the hive.auto.convert.join to true.
  2. Try changing the configuration of Java Heap Size and Garbage Collection reference Link

  3. Change the execution engine to Tez using set hive.execution.engine=tez To check currently set engine use hive.execution.engine.

Other Hive performance configuration tips can be found in the Link

Please let me know the results.

Answers 2

performance improvement by changinge these configuration parameters

-- enable cost based optimizer

set hive.cbo.enable=true;

set hive.compute.query.using.stats=true;

set hive.stats.fetch.column.stats=true;

set hive.stats.fetch.partition.stats=true;

--collects statistics

analyze table compute statistics for columns;

--enable vectorization of queries.

set hive.vectorized.execution.enabled = true;

set hive.vectorized.execution.reduce.enabled = true;

Read More

Modifing metadata from existing phAsset seems not working

Leave a Comment

In my App I want to make it possible, that the user sets an StarRating from 0 to 5 for any Image he has in his PhotoLibrary. My research shows, that there are a couple of ways to get this done:

Save the exif metadata using the new PHPhotoLibrary

Swift: Custom camera save modified metadata with image

Writing a Photo with Metadata using Photokit

Most of these Answers were creating a new Photo. My snippet now looks like this:

let options = PHContentEditingInputRequestOptions() options.isNetworkAccessAllowed = true  self.requestContentEditingInput(with: options, completionHandler: {             (contentEditingInput, _) -> Void in      if contentEditingInput != nil {          if let url = contentEditingInput!.fullSizeImageURL {             if let nsurl = url as? NSURL {                 if let imageSource = CGImageSourceCreateWithURL(nsurl, nil) {                     var imageProperties = CGImageSourceCopyPropertiesAtIndex(imageSource, 0, nil) as Dictionary?                     if imageProperties != nil {                         imageProperties![kCGImagePropertyIPTCStarRating] = rating as AnyObject                          let imageData = NSMutableData(contentsOf: url)                         let image = UIImage(contentsOfFile: url.path)                          let destination = CGImageDestinationCreateWithData(imageData!, CGImageSourceGetType(imageSource)!, 1, nil)                          CGImageDestinationAddImage(destination!, image!.cgImage!, imageProperties! as CFDictionary)                          var contentEditingOutput : PHContentEditingOutput? = nil                          if CGImageDestinationFinalize(destination!) {                             let archievedData = NSKeyedArchiver.archivedData(withRootObject: rating)                             let identifier = "com.example.starrating"                             let adjustmentData = PHAdjustmentData(formatIdentifier: identifier, formatVersion: "1.0", data: archievedData)                              contentEditingOutput = PHContentEditingOutput(contentEditingInput: contentEditingInput!)                             contentEditingOutput!.adjustmentData = adjustmentData                             if imageData!.write(to: contentEditingOutput!.renderedContentURL, atomically: true) {                                 PHPhotoLibrary.shared().performChanges({                                     let request = PHAssetChangeRequest(for: self)                                     request.contentEditingOutput = contentEditingOutput                                 }, completionHandler: {                                     success, error in                                     if success && error == nil {                                         completion(true)                                     } else {                                         completion(false)                                     }                                 })                             }                         } else {                             completion(false)                         }                      }                 }             }         }     } }) 

Now when I want to read the metadata from the PHAsset I request the ContentEditingInput again and do the following:

if let url = contentEditingInput!.fullSizeImageURL {     if let nsurl = url as? NSURL {         if let imageSource = CGImageSourceCreateWithURL(nsurl, nil) {             if let imageProperties = CGImageSourceCopyPropertiesAtIndex(imageSource, 0, nil) as Dictionary? {                  if let starRating = imageProperties[kCGImagePropertyIPTCStarRating] as? Int {                     rating = starRating                 }             }         }     } } 

But I never get my rating because it says that the value of imageProperties[kCGImagePropertyIPTCStarRating] is nil.

I also tried the examples from the Answers I posted above, but I always get the same result.

I hope anybody knows, what I can do to change the Metadata.

Also, how can I change the Metadata from an PHAsset with the MediaType .video? I tried to achieve that through the AVAssetWriter and AVExportSession Objects, but in both cases it does not work. Here what I tried for Videos:

var exportSession = AVAssetExportSession(asset: asset!, presetName: AVAssetExportPresetPassthrough) exportSession!.outputURL = outputURL exportSession!.outputFileType = AVFileTypeQuickTimeMovie exportSession!.timeRange = CMTimeRange(start: start, duration: duration)  var modifiedMetadata = asset!.metadata  let metadataItem = AVMutableMetadataItem() metadataItem.keySpace = AVMetadataKeySpaceQuickTimeMetadata metadataItem.key = AVMetadataQuickTimeMetadataKeyRatingUser as NSCopying & NSObjectProtocol metadataItem.value = rating as NSCopying & NSObjectProtocol  modifiedMetadata.append(metadataItem)  exportSession!.metadata = modifiedMetadata   exportSession!.exportAsynchronously(completionHandler: {     let status = exportSession?.status     let success = status == AVAssetExportSessionStatus.completed     if success {         do {             let sourceURL = urlAsset.url             let manager = FileManager.default             _ = try manager.removeItem(at: sourceURL)             _ = try manager.moveItem(at: outputURL, to: sourceURL)         } catch {             LogError("\(error)")             completion(false)         }     } else {         LogError("\(exportSession!.error!)")         completion(false)     } }) 

1 Answers

Answers 1

Sorry this isn't a full answer but it covers one part of your question. I noticed you are placing the StarRating in the wrong place. You need to place it in a IPTC dictionary. Also the properties data is stored as strings. Given you have the imageProperties you can add the star rating as follows and read it back using the following two functions

func setIPTCStarRating(imageProperties : NSMutableDictionary, rating : Int) {     if let iptc = imageProperties[kCGImagePropertyIPTCDictionary] as? NSMutableDictionary {         iptc[kCGImagePropertyIPTCStarRating] = String(rating)     } else {         let iptc = NSMutableDictionary()         iptc[kCGImagePropertyIPTCStarRating] = String(rating)         imageProperties[kCGImagePropertyIPTCDictionary] = iptc     } }   func getIPTCStarRating(imageProperties : NSMutableDictionary) -> Int? {     if let iptc = imageProperties[kCGImagePropertyIPTCDictionary] as? NSDictionary {         if let starRating = iptc[kCGImagePropertyIPTCStarRating] as? String {             return Int(starRating)         }     }     return nil } 

One other point you are creating an unnecessary UIImage. If you use CGImageDestinationAddImageFromSource() instead of CGImageDestinationAddImage() you can use the imageSource you created earlier instead of loading the image data into a UIImage.

Read More

Wednesday, June 28, 2017

Tree view header inside add the dropdown menu in odoo 8 using js and python

Leave a Comment

Tree view header inside add the dropdown menu in odoo 8 using js and python


I am use odoo-8. My question is how to add the dropdown menu above side on tree view in odoo8 using JS and python.

And I want to dynamically show the all the category of products in this drop-down menu.

And when i click on the particular category so, tree view inside sort particular clickable types of products.

i.e

Suppose click on the Mobile category from drop-down menu so tree view inside show only Mobile types of products.

Like I give the example in below image, enter image description here

I want to full solution of this question.
Note: If any query so comment please.

1 Answers

Answers 1

You have to use t-extend in your template.xml code.

ControlPanel is template name of the controlpanel that placed above tree view.

Add this code inside of your template.xml code.

<t t-name="your_template_name" t-extend="ControlPanel">     <t t-jquery=".o_cp_right" t-operation="after">         <-- Add your Drop down list here -->     </t> </t> 

o_cp_right is class after that you can add your dropdown field. You can set t-operation as append, inner, after, replace as per your requirment.

Read More

c# Make some tasks asynchronous

Leave a Comment

My Code:

ConcurrentQueue<string> concurrentQueue = new ConcurrentQueue<string>(); private void Form1_Load(object sender, EventArgs e) {   try {     var task1 = Task.Run(() => GetMessages());     var task2 = Task.Run(() => GetOrderBookData());     UpdateOrderBook();   } catch(Exception ex)   { MessageBox.Show(ex.Message); }  }  private void GetMessages() {   var w = new WebSocketConfiguration(); //class to connect o websocket to get messages   w.OnWSOpen += w_OnWSOpen;   w.OnWSMessage += w_OnWSMessage;   w.OnWSError += w_OnWSError;   w.OnWSClose += w_OnWSClose;   w.SetWebSocketSharpEvents(); // connect to websocket }  void w_OnWSMessage(string message) {   this.Invoke(new MethodInvoker(delegate()   {     listBox1.Items.Add(message);     concurrentQueue.Enqueue(message);     // To refresh the GUI simultaneously as the background process progresses     Application.DoEvents();                    }));             }  private void UpdateOrderBook() {   if (!concurrentQueue.IsEmpty) {     string jsonString;     while (concurrentQueue.TryDequeue(out jsonString))     {     }   } } 

Edit:

private void GetOrderBookData() {   var OrderList = new List<string>();    // Add items using Add method    OrderList.Add("order1");   OrderList.Add("order2");   OrderList.Add("order3");   OrderList.Add("order4");   OrderList.Add("order5");   dgOrders.DataSource = OrderList; } 

In my code UpdateOrderBook is called first and then GetMessages() is called. I want GetMessages() to keep running and once started it should call GetOrderBookData() to fill the values in a grid. Now I want to read messages from the queue and update the grid by calling UpdateOrderBook().

How do I make it asynchronous?

EDIT:

GetMessages() will send me more orders which I have to add/delete in the grid.

EDIT2

Steps: I want to call

(1) GetMessages() this will keep bringing the messages in separate thread

(2) Once the messages starts coming in (1) then call the GetOrderData() this will fetch the order details and insert them in datagrid and the its task is finished here

(3) UpdateOrderBook() this will now update the existing datagrid (which already has some data from step 2). Here in this method I want to check the queue and have to iterate the concurrentqueue then do some processing and then insert that record in the existing grid.

Hope this is clear now. The two process (1,3) would run asynchronously but for the first time it should process in above order.

I got stuck in making them run asynchronously.

4 Answers

Answers 1

Asynchronous methods that you define by using async and await are referred to as async methods. The good thing is that you can rewrite your code to tell the compiler to treat those methods as async. Now, you should be aware that if you don't call the await method inside an asynchronous methods it will not tell the compiler to stop the execution on that point and continue later. They good news is that you can execute the await over your method even if they return just void(Task)

What about this.

TaskFactory.StartNew Method (Action) Creates and starts a task. Parameters:action Type: System.Action The action delegate to execute asynchronously.Documentation Here

        Task task1 = Task.Factory.StartNew(GetMessages);         Task task2 = Task.Factory.StartNew(GetOrderBookData).ContinueWith(t => UpdateOrderBook());          Task.WhenAll(task1, task2).Wait(); 

This is another approach too

void Main() {             var cancelSource = new CancellationTokenSource();              Action action1 = async () =>             {                 await GetMessages();             };              Action action2 = async () =>             {                 //wait for filling the GetOrderBookData                 await GetOrderBookData();                 //on this point you should have the data to refresh on the grid                 UpdateOrderBook();             };              Task task1 = Task.Factory.StartNew(action1, cancelSource.Token);             Task task2 = Task.Factory.StartNew(action2, cancelSource.Token);              //when you want to cancel the task just call Cancel over cancelSource  => cancelSource.Cancel();              Task.WhenAll(task1, task2).Wait(); }  //Be sure to mark your methods with the async keyword private async Task UpdateOrderBook() { ... }  private async Task GetOrderBookData() {  .... }  private async Task GetMessages() {  .... } 

Answers 2

Maybe this will help:

using System; using System.Collections.Concurrent; using System.Collections.Generic; using System.Threading; using System.Threading.Tasks; using System.Windows.Forms;  namespace WindowsFormsApplication1 {     public partial class Form1 : Form     {         ConcurrentQueue<string> messageQueue = new ConcurrentQueue<string> ();          public Form1 ()         {             InitializeComponent ();         }          private async void Form1_Load (object sender, EventArgs e)         {             var task1 = GetMessages ();             var task2 = Task.Factory.StartNew (GetOrderBookData);              dgOrders.DataSource = await task2;              await Task.WhenAll (task1, task2);              UpdateOrderBook ();         }           public async Task GetMessages ()         {             for (int i = 0; i < 10; i++)             {                 string message = await Task.Run (() => GetMessage (i));                  messageQueue.Enqueue (message);                 listBox1.Items.Add (message);             }         }           private string GetMessage (int id)         {             Thread.Sleep (500); // simulate work             return "some-message-" + id;         }           private IReadOnlyList<string> GetOrderBookData ()         {             Thread.Sleep (2000); // simulate work              return new List<string> { "order 1", "order 2", "order 3" };         }           private void UpdateOrderBook ()         {             string message = null;              string jsonString;              while (messageQueue.TryDequeue (out jsonString))             {                 message += jsonString + "\r\n";             }              MessageBox.Show (message);         }     } } 

Answers 3

Try this.

    private void Form1_Load(object sender, EventArgs e)     {         try         {             //get messages asynchronously             Task getMessagesTask = new Task(GetMessages);             getMessagesTask.Start();              GetOrderBookData();              Task updateTask = new Task(UpdateOrderBook);              //check if concurrent queue has items             if(!concurrentQueue.IsEmpty)             {                 updateTask.Start();             }          }         catch (Exception ex)         { MessageBox.Show(ex.Message); }     } 

Answers 4

To make this code asynchronous you have to do something like this:

private async void Form1_Load(object sender, EventArgs e) {     var t1 = Task.Run(() => GetMessages());     var t2 = Task.Run(() => GetOrderBookData());     var t3 = Task.Run(() => UpdateOrderBook());     await Task.WhenAll(t1, t2, t3); } 

Now, there is a still a problem with your code in that it seems that UpdateOrderBook is dependent on the result of GetMessages. It seems that you are doing some sort of wait and check type loop in UpdateOrderBook. That's not a great idea.

Also the code for GetOrderBookData isn't even in your question.

It would be awesome to have the three methods fully implemented and posted in your question so that we can see how this all hangs together and then we could provide a better solution.

Read More

WkHtmlToPdf grey border around images

Leave a Comment

I'm creating a PDF using the Wkhtmltopdf library.

Everything is working fine, except one little detail: I overlap two images with transparent background to another background image.

This is the result:

The background image is the one with the trees and the sky, each character is in a different image with transparent background. So the result is quite perfect except for the grey border around each person.

Is it possible to remove it?

0 Answers

Read More

Matplotlib animation not showing

Leave a Comment

When I try this on my computer at home, it works, but not on my computer at work. Here's the code

import numpy as np import matplotlib.pyplot as plt import matplotlib.animation as animation import sys import multiprocessing   def update_line(num, gen, line):     data = gen.vals_queue.get()     data = np.array(data)     line.set_data(data[..., :num])     return line,   class Generator(multiprocessing.Process):     def __init__(self):         self.vals = [[], []]         super(Generator, self).__init__()         self.vals_queue = multiprocessing.Queue()      def run(self):         while True:             self.vals[0].append(np.random.rand())             self.vals[1].append(np.random.rand())             self.vals_queue.put(self.vals)   if __name__ == '__main__':     gen = Generator()     gen.start()     fig1 = plt.figure()     l, = plt.plot([], [], 'r-')     plt.xlim(0, 1)     plt.ylim(0, 1)     plt.xlabel('x')     plt.title('test')     print 11111111111111     sys.stdout.flush()     line_ani = animation.FuncAnimation(fig1, update_line, frames=None, fargs=(gen, l),                                        interval=50, blit=True, repeat=False)     print 222222222222222222222222     sys.stdout.flush()     plt.show()     print 3333333333333333333333333     sys.stdout.flush() 

And the output I see is

11111111111111 222222222222222222222222 3333333333333333333333333 

The application does not exit, it just hangs there, but no figure pops up. I run it from Linux terminal. My version of matplotlib is matplotlib-2.0.0-1.x86_64

Also, I've got this at work (problematic one)

CentOS Linux release 7.2.1511 (Core)  echo $SHELL /bin/bash echo $BASH_VERSION 4.2.46(1)-release Python 2.7.12 

2 Answers

Answers 1

It is really hard to reproduce this problem, so I'll try to give some general advises and try to guess the actual root of the problem.

First of all, it is in your best interest to use virtualenvs, if you aren't using them already. You will have a requirements.txt file in your project and will freeze the requirements from your home computer (the one that works) into requirements.txt, then will create a new virtualenv on the computer at work and finally install the requirements. That way you will be sure that you have the same versions of all packages on both computers.

After that you should try and see if it works. If it doesn't please try these things and provide more details:

  1. Do you see any errors or warnings when you run it on the computer at work?
  2. Can you do very basic plots using matplotlib? Like this one:

    import matplotlib.pyplot as plt plt.plot([1, 2, 3, 4]) plt.ylabel('some numbers') plt.show()

  3. If the example from 2 doesn't work, try to replace plt.show() with plt.savefig('numbers.png') and see if the figure is successfully saved. If that's the case, then you have some problems with matplotlib's interactivity. If you can't see a file named numbers.png, then probably there is something wrong with matplotlib's installation in general, not just the animation part. Or maybe with the installation of some package matplotlib relies on, like Tkinter, for instance.

Instead of going into further hypothetical scenarios, I'll stop here and wait for more details.

p.s. Links you may find useful if there is problem with showing the generated plots/animations in a window:

How can I set the 'backend' in matplotlib in Python?

http://matplotlib.org/faq/usage_faq.html#what-is-a-backend

Answers 2

It seems that the library is exposed to some changes or it is not installed properly. Simply update the library as below,

For Ubuntu/Debian OS open a terminal and type:

                    sudo apt-get install python-matplotlib 

For windows open a command line and type:

                    python -m pip install -U pip setuptools                     python -m pip install matplotlib 

See if that works.

Source: https://matplotlib.org/users/installing.html

Read More

How failover works on Google Cloud SQL?

Leave a Comment

I intend to connect a PHP app (from a server outside Googel Cloud Platform) to Google Cloud SQL. I want to know how can I design the app to failover its database properly.

According to the manual:

When a zonal outage occurs and your master fails over to your failover replica, any existing connections to the instance are closed. However, your application can reconnect using the same connection string or IP address; you do not need to update your application after a failover.

It appears everything is happenning automatically behind the scenes but what if the IP address of the database is out or times out?

1 Answers

Answers 1

Google doesn't reveal here what back-end is using to provide database service but my assumption is:

  • It is using a virtual IP that are routing traffic to master connector, therefore when master fails the backup would start to advertise the virtual IP, this would cause your TCP connection fails.

The comment means if your mysqli.reconnect in php.ini must be enabled to allow your PHP code automatically connect to the backup MySQL connector when failure occurs. read here

If your mysqli.reconnect is enabled then you don't have to worry about anything.

Edit: in reply to question raised regarding SQL Transaction -- Of course it will be a mess if the SQL transaction code is not written with a precaution that connection may disconnect at any time during transaction, such scenario must be handled in code. and when auto-commit is enabled and you are not within a transaction a simple auto-reconnect would solve almost any other scenario.

Read More

Logging lots of Android sensor data

Leave a Comment

I have an Android application that is logging several Android sensors at approximately 100Hz. So if i am logging 10 sensors, I am writing about 3000 data points per second to a file (each sensor typically has 3 entries). Now the problem is that i want to minimize the effect of this writing on the rest of the app. Specifically, I do not want the logging to slow down event delivery... i want to make sure that i am getting events as soon as they happen, and not with a delay (i know there will always be some delay because Android is not realtime and because of the "pull" nature of the Sensor event framework).

Next, I will outline my approach, which does not appear to be working well. I would like suggestions for how to improve.

My current procedure is this...

For each sensor, i create a separate thread with a BlockingQueue of events to log. Inside the thread, I have a while loop that pulls from the queue and does the file writing using a buffered writer. When the sensor manager delivers a new SensorEvent, the event is put in the appropriate queue (thereby triggering the file IO on the other thread) so as not to delay the main thread on which SensorEvents are delivered.

I want to be getting the events as soon as they occur, so it is important that i do not introduce any delays in the Sensor framework. If, for instance, I did the file IO directly in the onEvent callback, then i am worried that events could start piling up in the pipeline, and that they would then be out of date by the time they are finally delivered. The above approach mitigates these worries.

But there is another issue...

Despite the fact that the file IO occurs off of the sensor event delivery thread, at times the application still feels sluggish. That is, sometimes i will see events occur in rapid succession (e.g. 5 events are delivered within 1 ms of each other). This indicates that although the IO is not happening on the sensor delivery thread, the delivery thread is still getting delayed. A few reasons have been suggested to me:

  1. I am creating too many IO threads. Perhaps if i pushed all of the writing to a single thread I would increase the likelihood that the sensor delivery thread is alive when a new event comes in. In the current setup, it could be that all of the active threads are being used for file IO when an Event comes in, resulting in the events backing up until one of the writing events finishes.

  2. Currently, i am using flat file output, not a database. The benefits of using a database for retrieval are clear to me. What is not clear to me is whether i should also expect a database to be faster if I am only appending data to a file.... that is, i never need to read from the file or insert the data into a random place , I just literally append to the end of the file. It seems to me that a database cannot be any faster than standard file IO in that case. Or am I wrong?

  3. Others have suggested that the garbage collector is probably interfering with my threads and that the likely source of the problem is memory thrashing due to the large number of events that are being created.

From which angle should i approach this?

3 Answers

Answers 1

It's all guesswork until you attach a profiler and see what really happens. From general experience I'd say that points 1 and 3 are definitely valid. Point 2 less so; if a database is faster than appending to a file I think that's likely down to implementation quality and/or using C with native API's and you should be able to overcome the difference.

With regards to GC load, take a look at Looper (api) and Handler (api). Using these you can replace your BlockingQueue-based approach by one that generates no GC load at all. I wrote a blog post exploring this in some detail.

Furthermore, if you're using any/all of the Clean Code practices, it may be time for some judicious use of dirty code (field access, mutability) to lighten the memory churn.

With regards to IO threads: I'd say definitely scale down to a single one, and try to write the lines to file in batches, not one by one. A buffered Stream or Writer would probably do the trick on its own, so long as you avoid explicitly calling flush().

Answers 2

Yours is a subject that can hardly be evaluated if it is not tested, but I make some suggestions:

  • To use a single thread you could use a Handler and a Looper as proposed by Barend and when you have a new sensor reading a message is send to the this Handler, so there is the possibility of deleting old messages when you have several pending readings of the same sensor, that add some sort of anti-overflow protection, if the system can't process all readings some will be lost.

  • As you say Android is not real time, if the sensors are not read at regular times maybe you should attach a timestamp to each reading.

  • To avoid allocating & GC'ing variables or objects for each sensor reading you could use a pool (array) of variables previously allocated and each time you have a reading you use the following index. This would be fine for the case of sending messages because it would only have to send the index of the new reading in the message arguments.

I hope it helps.

Answers 3

https://docs.oracle.com/javase/7/docs/api/java/math/BigInteger.html#toByteArray()

Using above will save enough on IO to make your sensor code work. I've seen android do higher bit rates recording video. There shouldn't be a problem writing a file. 3000 elements per second Generous 20 bytes per row 480kbs.

Put your code into a service it will stay running when the app goes to the background while user checks instagram or goes looking for pokemon or the user presses the home button.

GC doesn't GC avoid final objects? IDK. Maybe I'm wrong but you should only be creating complex objects once. The service stays in memory running. not a problem.

five in one ms. This is normal for asycronous code. It's the nature of multitasking. Program had five events queued up and it processed them. You have to prioritize the tasks but don't put too much into your prioritizing or it will cause problems. For example if the write to file doesn't have enough priority the file write may never happen. for now keep all tasks at the same priority until you understand the profiler output.

Write to file. Maybe its going through encyption and onto the SDCard that would be bad and would take all kinds of CPU and cause even more problems.

Read More

How to make text in <meter> tag visible over the meter

Leave a Comment

Given the html

<meter value="0.55" high="0.999" optimum="1">   <span class="meter-value">0.5491</span> </meter> 

I would like text 0.5491 on top of the meter. I tried to style the text using usual CSS techniques, but it won't show at all. In inspector it just says width and height is 0 no matter how much I say things like

.meter-value {   display: block; width: 50px; height: 20px; } 

I made more attempts, these examples are simplified. I use Firefox for tests and compatibility is not pressing issue (inhouse project).

I much more prefer "ugly css" solution above "ugly html" solution.

Update: (to sum up some comments)

There seems to be some magic which makes content from meter invisible (including the content from meter::after, meter span::after and simillar components) in Firefox. So the question is if there is a css which can override this. Setting width, height and visibility did not help.

5 Answers

Answers 1

Make use of CSS3's ::after and add it to meter tag as follows.

meter::after {   content : "0.5491"; } 

This will show the text below the meter.

update

Since OP said that he want multiple elements, updated the code. Changed position from absolute to relative so the text will be always relative to the meter

In order to make it appear on the meter, style it using position:absolute and give top or margin-top and left as follows

meter{   width:100px; }  meter::after {  content : attr(value);  top:-17px;  left:40px;  position:relative; } 

For more reference on ::after, Visit MDN

Also using that, you can remove the span element.

Here, we make use of attr() function in css. Read about it in MDN

Try the below snippet

meter{    width:100px;  }    meter::after {   content : attr(value);   top:-17px;   left:40px;   position:relative;  }
<meter value="0.55" high="0.999" optimum="1">  </meter>  <meter value="0.45" high="0.999" optimum="1">  </meter>  <meter value="0.85" high="0.999" optimum="1">  </meter>  <meter value="0.95" high="0.999" optimum="1">  </meter>    <br />  <meter value="0.55" high="0.999" optimum="1">  </meter>  <meter value="0.45" high="0.999" optimum="1">  </meter>  <meter value="0.85" high="0.999" optimum="1">  </meter>  <meter value="0.95" high="0.999" optimum="1">  </meter>

The above code doesn't work on Firefox. It is a known issue that ::after and ::before pseudos work only in webkit browsers.

For firefox, try the following code (This is global. It will work on all browsers)

meter{    width:100px;  }  span{  }  span::after {   content : attr(data-value);   top:0px;   left:-70px;   position:relative;  }
<meter value="0.55" high="0.999" optimum="1">  </meter>  <span data-value="0.55"></span>  <meter value="0.45" high="0.999" optimum="1">  </meter>  <span data-value="0.45"></span>  <br />  <meter value="0.55" high="0.999" optimum="1">  </meter>  <span data-value="0.55"></span>  <meter value="0.45" high="0.999" optimum="1">  </meter>  <span data-value="0.45"></span>

Answers 2

There might be other way of doing this. I can see this works: add separate code and use that.

<meter value="0.55" high="0.999" optimum="1">   <span class="meter-value">0.5491</span> </meter> <span class="meter-value">0.5491</span> 

https://jsfiddle.net/f083nfm1/

Answers 3

Whelp, that's an interesting question. The only thing that I can think of is to use content property, used in ::before and ::after pseudo-selector.

Yet, the problem is the value, assuming if you have multiple meters. Well, I have a solution for it I think: the content's attr(...) keyword.

So you only have to provide a value as attribute, like the data-value here below.

<meter value="0.55" high="0.999" optimum="1" data-value="0.5491"></meter> 

Then it's simply adding and adjusting the positions of the added content.

But ... sadly it's not widely supported (yet). Only chrome I think...

meter::after {   content: attr(data-value);   position:relative;   left: 1.2em;   top: -1em;  }
<meter value="0.55" high="0.999" optimum="1" data-value="0.5491"></meter>    <meter value="0.22" high="0.999" optimum="1" data-value="0.2219"></meter>

Answers 4

adding ::before selector to css you can view the text on top of meter

CSS:

    meter::before {   content : "0.5491"; } 

HTML:

    <meter value="0.55" high="0.999" optimum="1">   <span class="meter-value">0.5491</span> </meter> 

Answers 5

I think this is the best solution, put in the value of the meter the data

meter:before{   content:attr(value); } 
Read More

How to define the Cache-Control header on an S3 object via a signed URL?

Leave a Comment

Following the instructions in this guide, I've managed to get uploads working via signed URLs. It looks something like this:

const s3 = new aws.S3();  const s3Params = {   Bucket: S3_BUCKET,   Key: fileName,   Expires: 60,   ContentType: fileType,   ACL: 'public-read',   CacheControl: 'public, max-age=31536000', };  s3.getSignedUrl('putObject', s3Params, (err, data) => {   // ... }); 

...except my CacheControl param (which I added myself; it isn't in the guide) does not seem to take effect. When I use the above code to generate a signed URL and upload something to it, the resulting object in S3 is served with no Cache-Control header.

What am I doing wrong?

1 Answers

Answers 1

You must send the Cache-Control header in the upload request, regardless of what you set during the signed URL generation.

Whether this is a bug or an intentional behaviour is questionable and beyond my ability to answer. The Cache-Control header, as you noticed, is part of the signed URL, but for whatever reason the information is completely ignored during the file upload, ie. not specifying a CacheControl property in the getSignedUrl() function still allows the client to set Cache-Control header to whatever value they choose.

If you need to have control over the Cache-Control header, then using the getSignedUrl() is most likely not appropriate for your use case.

Solution

AWS now supports a new signature scheme, called AWS Signature version 4 which allows full control over what the upload request may or may not contain, including which headers are sent and with what values.

The JavaScript SDK supports this new signature version: createPresignedPost().

A detailed example of how to generate this pre-signed POST policy and how the upload form should look like can be found directly on AWS's documentation.

Even though the example demonstrates the file upload via standard http upload <form> element, the principles can be applied to any client/consumer capable of performing HTTP communication.

Example

For completeness, here is the example (taken from AWS documentation page linked above) of how a pre-signed POST policy looks like:

{ "expiration": "2015-12-30T12:00:00.000Z",   "conditions": [     {"bucket": "sigv4examplebucket"},     ["starts-with", "$key", "user/user1/"],     {"acl": "public-read"},     {"success_action_redirect": "http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html"},     ["starts-with", "$Content-Type", "image/"],     {"x-amz-meta-uuid": "14365123651274"},     {"x-amz-server-side-encryption": "AES256"},     ["starts-with", "$x-amz-meta-tag", ""],      {"x-amz-credential": "AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request"},     {"x-amz-algorithm": "AWS4-HMAC-SHA256"},     {"x-amz-date": "20151229T000000Z" }   ] } 

This POST policy sets the following conditions on the request:

  • The upload must occur before midnight UTC on December 30, 2015.
  • The content can be uploaded only to the sigv4examplebucket. The bucket must be in the region that you specified in the credential scope (x-amz-credential form parameter), because the signature you provided is valid only within this scope.
  • You can provide any key name that starts with user/user1. For example, user/user1/MyPhoto.jpg.
  • The ACL must be set to public-read.
  • If the upload succeeds, the user's browser is redirected to http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html.
  • The object must be an image file.
  • The x-amz-meta-uuid tag must be set to 14365123651274.
  • The x-amz-meta-tag can contain any value.

Note that the list of conditions in this example is not exhaustive and CacheControl is supported. See the creating a POST policy document for what you can do with this.

Read More