MATLAB: Initialize and empty table with headers and concatenate it to another table

I'm getting this error:

Cannot concatenate the table variable 'epoch' because it is a cell in one table and a non-cell in another.

To reproduce the error:

A.epoch = [1,2,3]'; A.value = [10,20,30]'; 

Initialized empty table with headers:

Aclean = cell2table(cell(1,2), 'VariableNames', {'epoch', 'value'}); 

Vertically concatenate the 2 tables:

Aclean = vertcat(Aclean, struct2table(A)); 

What is the best way of concatenating tables with a for loop in Matlab?

1 Answers

Answers 1

If I understand right, you want to initialize Aclean before a loop, then concatenate data to it in every loop iteration. If so, you can simply initialize Aclean to the empty array:

Aclean = [];  A.epoch = [1,2,3].'; A.value = [10,20,30].'; Aclean = vertcat(Aclean, struct2table(A)); 

However, if you know in advance how many rows you will add to the table, it is better to preallocate the full table:

% N = number of rows A = array2table(zeros(N,2), 'VariableNames', {'epoch', 'value'}); A.epoch(1:3) = [1,2,3]; A.value(1:3) = [10,20,30]; 

It is cheaper to cut off unused rows from the bottom than to append in every iteration.

Yet another alternative is to work with a struct and convert it to a table after your loop. A struct is a built-in type, whereas a table is implemented as a custom class. The struct will therefore be more efficient to update.

Read More

Drag and drop issue in Chrome related to Windows scale (125%)

I have an issue with drag and drop on Chrome (v69.0.3497.100). Specifically, some of the drag and drop events are getting fired when Windows scaling is other than 100% even though they shouldn't be firing.

Check out stackblitz example, and try to drag "blue" rectangle over itself (just drag, move a little bit downwards and drop). If Windows scaling is set to 100% (browser zoom is 100% as well) then one event is fired (dragEnter) as expected (check the console). But, if Windows scaling is set to 125% (but browser zoom is still 100%) then three events are fired (two dragEnter and one dragLeave), and I expected only one event to be fired since the element was dragged and dropped on itself (as it was the case with 100% scale level).

It could be that since this is Windows zoom (and not browser's zoom) the left ("lightred") rectangle is larger that it appears, and it goes below right rectangle, and events are propagated to it, although I couldn't prove that since all elements have correct size in the inspector.

This doesn't seem to be happening in latest Firefox, IE or Edge.

Does anyone know why is this happening and how to fix it?

Thank you.

0 Answers

Read More

Register a package for the class generated from the proto file in the gradle by default , not in proto file

In my android app I use proto files. For example, I have proto file Stats.proto

syntax = "proto3"; package com.me.test;  message Stat {     string a = 1;     string b = 2;     string c = 3;     string d = 4; } 

I need to register package in the each proto file itself, and this is uncomfortable, because I have a lot of files. I want to register default package in gradle, for example, 'package com.me.test', which uses each file that I create. I found solution in javanano

nano {     proto {       // Selects between --java_out and --javanano_out, default is 'java'       mode 'javanano'       // Options added to --java_out or --javanano_out       option 'java_package=src/proto/simple-data.proto|my_package'       option 'java_outer_classname=src/proto/simple-data.proto|OuterName'       // Apply the 'grpc' plugin defined in protobufNativeCodeGenPluginDeps       plugin 'grpc' {         // Options added to --grpc_out         option 'nano=true'       }     } 

But I need to use javalite

protobuf {     protoc {         artifact = 'com.google.protobuf:protoc:3.0.0'     }     plugins {         javalite {             artifact = 'com.google.protobuf:protoc-gen-javalite:3.0.0'         }     }     generateProtoTasks {         all().each { task ->             task.builtins {                 remove java             }             task.plugins {                 javalite { }             }         }     } } 

And I want to have analogous logic

option 'java_package=src/proto/simple-data.proto|my_package' 

like javanano in javalite

Can I implement that?

1 Answers

Answers 1

According to the doc, and assuming javalite supports the same features than java (it's a long shot as I did not found any trace of documentation on this plugin), just try that :

protobuf {     protoc {         artifact = 'com.google.protobuf:protoc:3.0.0'     }     plugins {         javalite {             artifact = 'com.google.protobuf:protoc-gen-javalite:3.0.0'         }     }     generateProtoTasks {         all().each { task ->             task.builtins {                 remove java             }             task.plugins {                 javalite {                     option 'java_package=.......'                                  }             }         }     } } 

Read More

get position of the text inside a TextView

suppose i have the following text 'ADD TEST' inside TextView as shown below

as you can see the text inside the textView does not have the same width and height as textView .

what i want is to get the x,y position of text inside the textView

2 Answers

Answers 1

Y value

You can use textView.getTextSize() or textView.getPaint().getTextSize() to get the actual used text size in pixels (as Float).

Next, we need the total height of the text view, which we can find as follows:

textView.measure(0, 0); // We must call this to let it calculate the heights int height = textView.getMeasuredHeight(); 

However, the final size that we need can also have decimals. So lets make it a float for more precision:

float totalHeight = (float) height; 

Now that we know the values, we can calculate the y value of the text inside the view:

// The spacing between the views is `totalHeight - textSize` // We have a spacing at the top and the bottom, so we divide it by 2 float yValue = (totalHeight - textSize) / 2 

X value

Furthermore, the xValue is just the x value of the text view itself when using android:includeFontPadding="false".

Answers 2

Take a look at a couple of Paint methods: getTextBounds() and measureText. We can use these to determine the offset of the text within the TextView. Once the offset within the TextView is determined, we can add that to the location of the TextView itself to determine the screen coordinates of the text if that is desired.

I have also found the article "Android 101: Typography" to be useful in understanding some of the complexities of typography.

The following example finds the bounds of the text within three TextViews and draws a rectangle around the text. The rectangle contains the (x, y) coordinates of the text within the TextView.

activity_main.xml
A simple layout for demonstration.

<android.support.constraint.ConstraintLayout     android:id="@+id/layout"     android:layout_width="match_parent"     android:layout_height="match_parent"     tools:context=".MainActivity">      <TextView         android:id="@+id/textView1"         android:layout_width="wrap_content"         android:layout_height="wrap_content"         android:layout_marginTop="24dp"         android:background="@android:color/holo_blue_light"         android:padding="24dp"         android:text="Hello World"         android:textColor="@android:color/black"         android:textSize="50sp"         app:layout_constraintLeft_toLeftOf="parent"         app:layout_constraintRight_toRightOf="parent"         app:layout_constraintTop_toTopOf="parent" />      <TextView         android:id="@+id/textView2"         android:layout_width="wrap_content"         android:layout_height="wrap_content"         android:layout_marginTop="24dp"         android:background="@android:color/holo_blue_light"         android:padding="24dp"         android:text="Hello Worldly"         android:textColor="@android:color/black"         android:textSize="50sp"         app:layout_constraintLeft_toLeftOf="parent"         app:layout_constraintRight_toRightOf="parent"         app:layout_constraintTop_toBottomOf="@id/textView1" />      <TextView         android:id="@+id/textView3"         android:layout_width="wrap_content"         android:layout_height="wrap_content"         android:layout_marginTop="24dp"         android:background="@android:color/holo_blue_light"         android:padding="24dp"         android:text="aaaaaaaaaa"         android:textColor="@android:color/black"         android:textSize="50sp"         app:layout_constraintLeft_toLeftOf="parent"         app:layout_constraintRight_toRightOf="parent"         app:layout_constraintTop_toBottomOf="@id/textView2" />  </android.support.constraint.ConstraintLayout> 

MainActivity.java

public class MainActivity extends AppCompatActivity {      @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         setContentView(R.layout.activity_main);          drawTextBounds((TextView) findViewById(R.id.textView1));         drawTextBounds((TextView) findViewById(R.id.textView2));         drawTextBounds((TextView) findViewById(R.id.textView3));     }      private void drawTextBounds(TextView textView) {         // Force measure of text pre-layout.         textView.measure(0, 0);         String s = (String) textView.getText();          // bounds will store the rectangle that will circumscribe the text.         Rect bounds = new Rect();         Paint textPaint = textView.getPaint();          // Get the bounds for the text. Top and bottom are measured from the baseline. Left         // and right are measured from 0.         textPaint.getTextBounds(s, 0, s.length(), bounds);         int baseline = textView.getBaseline();         bounds.top = baseline + bounds.top;         bounds.bottom = baseline + bounds.bottom;         int startPadding = textView.getPaddingStart();         bounds.left += startPadding;          // textPaint.getTextBounds() has already computed a value for the width of the text,          // however, Paint#measureText() gives a more accurate value.         bounds.right = (int) textPaint.measureText(s, 0, s.length()) + startPadding;          // At this point, (x, y) of the text within the TextView is (bounds.left, bounds.top)         // Draw the bounding rectangle.         Bitmap bitmap = Bitmap.createBitmap(textView.getMeasuredWidth(),                                             textView.getMeasuredHeight(),                                             Bitmap.Config.ARGB_8888);         Canvas canvas = new Canvas(bitmap);         Paint rectPaint = new Paint();         rectPaint.setColor(Color.RED);         rectPaint.setStyle(Paint.Style.STROKE);         rectPaint.setStrokeWidth(1);         canvas.drawRect(bounds, rectPaint);         textView.setForeground(new BitmapDrawable(getResources(), bitmap));     } } 

Read More

gradle: disable creation of the build folder at root in multi-projects application

My gradle project is a multi-project structure. It consists of two subprojects. My build.gradle in root project as following:

allprojects {   apply plugin: 'eclipse' }  subprojects {   apply plugin: 'java'   apply plugin: 'maven-publish'    publishing {     repositories {       maven {         name 'myMaven'         def suffix = project.version.endsWith("SNAPSHOT") ? "snapshots" : "releases"         url baseUrl + suffix       }     }      publications {       core(MavenPublication) {         artifactId project.name         artifact jar       }     }   } }  project(':projectA') {   ext {     sharedManifest = manifest {       attributes("Implementation-Title": project.name,                  "Implementation-Version": 1.0 )     }   }   jar {     manifest = project.manifest {       from sharedManifest     }   } }  project('projectB') {   ext {     sharedManifest = manifest {       attributes("Implementation-Title": project.name,                  "Implementation-Version": 2.0 )     }   }   jar {     manifest = project.manifest {       from sharedManifest     }   } } 

After gradle build, I got three build folders. one at the root directory and other two in the respective subprojects. The root project is just an empty folder without any source code, it is just a multi-project container.

Now I need to eliminate the creation of root build folder since it is of no use. I search on the net and in the gradle docs/forum but did not get any hit.

Is there any way so that gradle stop creating build folder at root level?

1 Answers

Answers 1

Who is responsible for the build dir creation ?

According to Mark Vieira, a Gradle core dev, each task is responsible for the creation of the ouput dir as long as it creates output.

So basically, it your root project does have a build dir, it means it's not empty and something is deliberately writing into it.

In my job, I'm managing a big project constituted by a root project, which handles many .gradle files, settings files etc. and many subprojects with the actual code. When I run gradle clean build, all subprojects get their own build dir with libs, binaries etc. but the root project does not end up with any build dir because no task whatsoever writes output in the root build dir.

What can I do to remove the root build dir, whatever task creating it ?

Just set up a buildFinished hook only for the root project

gradle.buildFinished {     project.buildDir.deleteDir() } 

Read More

Python lagged series to Pyspark

I am trying to do adapt this Python code in pyspark:

from statsmodels.tsa.tsatools import lagmat  def lag_func(data,lag):     lag = lag     X = lagmat(data["diff"], lag)     lagged = data.copy()     for c in range(1,lag+1):         lagged["lag%d" % c] = X[:, c-1]     return lagged  def diff_creation(data):     data["diff"] = np.nan     data.ix[1:, "diff"] = (data.iloc[1:, 1].as_matrix() - data.iloc[:len(data)-1, 1].as_matrix())     return data 

The result is a dataframe with lagged columns.

I tried something like that:

class SerieMaker(Transformer):     def __init__(self, inputCol='f_qty_recalc', outputCol='serie', dateCol='dt_ticket_sale', idCol= ['id_store', 'id_sku'], serieSize=30):         self.inputCol = inputCol         self.outputCol = outputCol         self.dateCol = dateCol         self.serieSize = serieSize         self.idCol = idCol      def _transform(self, df):         window = Window.partitionBy(self.idCol).orderBy(self.dateCol)         series = []             df = df.withColumn('filled_serie', F.lit(0))          """ 30 days lag"""          for index in reversed(range(0, self.serieSize)):             window2 = Window.partitionBy(self.idCol).orderBy(self.dateCol).rowsBetween((self.serieSize - index), self.serieSize)             col_name = (self.outputCol + '%s' % index)             series.append(col_name)             df = df.withColumn(col_name, F.when(F.isnull(F.lag(F.col(self.inputCol), index).over(window)),                                                  F.first(F.col(self.inputCol),                                                          ignorenulls=True).over(window2)).otherwise(F.lag(F.col(self.inputCol),                                                                                                           index).over(window)))             df = df.withColumn('filled_serie', F.when(F.isnull(F.lag(F.col(self.inputCol), index).over(window)),                                                        (F.col('filled_serie') + 1)).otherwise(F.col('filled_serie')))             df = df.withColumn('rank', F.rank().over(window))             return df.withColumn(self.outputCol, F.col(*series)) 

My df looks like:

  id_sku|id_store|     dt_ticket_sale|f_qty_recalc|prc_sku|sales| +------------+--------+-------------------+------------+-------+-----+ |    514655.0|    1090|2017-12-20 00:00:00|           1|   1.23| 1.23| |    823259.0|     384|2017-12-20 00:00:00|           1|   2.79| 2.79| 

My expected output is some lag of fqty_recalc and at the beginning idsku idstore and date (not shown there):

    diff    lag1    lag2    lag3    lag4    lag5    lag6    lag7    lag8    lag9    ... lag20   lag21   lag22   lag23   lag24   lag25   lag26   lag27   lag28   lag29 0   NaN 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1   0.0 NaN 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 

0 Answers

Read More

How to fix or work around apparent bug in plotly's event_data(“plotly_hover”) when interrogating 3d surface plots

I've produced an app where the aim is to combine four surfaces of values on a common 3D plane, with corresponding subplots that show cross-sections of z ~ y and z ~ x. To do this I'm trying to use event_data("plotly_hover") to extract the x and y values of the surface.

However, the maximum x value recorded by event_data("plotly_hover") is truncated at around 38, whereas the maximum x value is 80. The tooltips for the surface itself, however, are correct.

This is shown in the two figures: the first shows both the tooltip and event_data() output where x < 38, both of which are correct; and the latter shows the tooltip and event_data where x > 38. The tooltip correctly describes the corresponding values, but the event_data output is stuck at the last position where x == 38.

The code is reproduced below (much of which is about the construction of the tooltip). Any suggestions for why event_data is not working correctly in this instance, and suggested solutions (either using event_data or a work-around) are much appreciated.

# # This is a Shiny web application. You can run the application by clicking # the 'Run App' button above. # # Find out more about building applications with Shiny here: # #    http://shiny.rstudio.com/ # library(tidyverse) library(shiny) library(RColorBrewer) library(plotly) read_csv("https://github.com/JonMinton/housing_tenure_explorer/blob/master/data/FRS%20HBAI%20-%20tables%20v1.csv?raw=true") %>%  #read_csv("data/FRS HBAI - tables v1.csv") %>%    select(     region = regname, year = yearcode, age = age2, tenure = tenurename, n = N_ten4s, N = N_all2   ) %>%    mutate(     proportion = n / N   ) -> dta   regions <- unique(dta$region)  tenure_types <- unique(dta$tenure)   # Define UI for application that draws a histogram ui <- fluidPage(     # Application title    titlePanel("Minimal example"),     # Sidebar with a slider input for number of bins     sidebarLayout(       sidebarPanel(          sliderInput("bins",                      "Number of bins:",                      min = 1,                      max = 50,                      value = 30)       ),        # Show a plot of the generated distribution       mainPanel(          plotlyOutput("3d_surface_overlaid"),          verbatimTextOutput("selection")       )    ) )  # Define server logic required to draw a histogram server <- function(input, output) {    output$`3d_surface_overlaid` <- renderPlotly({     # Start with a fixed example       matrixify <- function(X, colname){       tmp <- X %>%          select(year, age, !!colname)       tmp %>% spread(age, !!colname) -> tmp       years <- pull(tmp, year)       tmp <- tmp %>% select(-year)       ages <- as.numeric(names(tmp))       mtrx <- as.matrix(tmp)       return(list(ages = ages, years = years, vals = mtrx))     }       dta_ss <- dta %>%        filter(region == "UK") %>%        select(year, age, tenure, proportion)       surface_oo <- dta_ss %>%        filter(tenure == "Owner occupier") %>%        matrixify("proportion")      surface_sr <- dta_ss %>%        filter(tenure == "Social rent") %>%        matrixify("proportion")      surface_pr <- dta_ss %>%        filter(tenure == "Private rent") %>%        matrixify("proportion")      surface_rf <- dta_ss %>%        filter(tenure == "Care of/rent free") %>%        matrixify("proportion")       tooltip_oo <- surface_oo      tooltip_sr <- surface_sr      tooltip_pr <- surface_pr      tooltip_rf <- surface_rf      custom_text <- paste0(       "Year: ", rep(tooltip_oo$years, times = length(tooltip_oo$ages)), "\t",       "Age: ", rep(tooltip_oo$ages, each = length(tooltip_oo$years)), "\n",       "Composition: ",        "OO: ", round(tooltip_oo$vals, 2), "; ",       "SR: ", round(tooltip_sr$vals, 2), "; ",       "PR: ", round(tooltip_pr$vals, 2), "; ",       "Other: ", round(tooltip_rf$vals, 2)     ) %>%        matrix(length(tooltip_oo$years), length(tooltip_oo$ages))      custom_oo <- paste0(       "Owner occupation: ", 100 * round(tooltip_oo$vals, 3), " percent\n",       custom_text     ) %>%        matrix(length(tooltip_oo$years), length(tooltip_oo$ages))      custom_sr <- paste0(       "Social rented: ", 100 * round(tooltip_sr$vals, 3), " percent\n",       custom_text     ) %>%        matrix(length(tooltip_oo$years), length(tooltip_oo$ages))      custom_pr <- paste0(       "Private rented: ", 100 * round(tooltip_pr$vals, 3), " percent\n",       custom_text     ) %>%        matrix(length(tooltip_oo$years), length(tooltip_oo$ages))      custom_rf <- paste0(       "Other: ", 100 * round(tooltip_rf$vals, 3), " percent\n",       custom_text     ) %>%        matrix(length(tooltip_oo$years), length(tooltip_oo$ages))      n_years <- length(surface_oo$years)     n_ages <- length(surface_oo$ages)      plot_ly(       showscale = F     ) %>%        add_surface(         x = ~surface_oo$ages, y = ~surface_oo$years, z = surface_oo$vals,         name = "Owner Occupiers",         opacity = 0.7,         colorscale = list(           c(0,1),           c('rgb(255,255,0)' , 'rgb(255,255,0)')         ),         hoverinfo = "text",         text = custom_oo        ) %>%        add_surface(         x = ~surface_sr$ages, y = ~surface_sr$years, z = surface_sr$vals,         name = "Social renters",         opacity = 0.7,         colorscale = list(           c(0,1),           c('rgb(255,0,0)' , 'rgb(255,0,0)')         ),         hoverinfo = "text",         text = custom_sr        ) %>%        add_surface(         x = ~surface_pr$ages, y = ~surface_pr$years, z = surface_pr$vals,         name = "Private renters",         opacity = 0.7,         colorscale = list(           c(0,1),           c('rgb(0,255,0)' , 'rgb(0,255,0)')         ),         hoverinfo = "text",         text = custom_pr        ) %>%        add_surface(         x = ~surface_rf$ages, y = ~surface_rf$years, z = surface_rf$vals,         name = "Other",         opacity = 0.7,         colorscale = list(           c(0,1),           c('rgb(0,0,255)' , 'rgb(0,0,255)')         ),         hoverinfo = "text",         text = custom_rf         ) %>%        layout(         scene = list(           aspectratio = list(             x = n_ages / n_years, y = 1, z = 0.5           ),           xaxis = list(             title = "Age in years"           ),           yaxis = list(             title = "Year"           ),           zaxis = list(             title = "Proportion"           ),           showlegend = FALSE         )      )    })    output$selection <- renderPrint({     s <- event_data("plotly_hover")     if (length(s) == 0){       "Move around!"     } else {       as.list(s)     }    })  }  # Run the application  shinyApp(ui = ui, server = server) 

1 Answers

Answers 1

Indeed there is something strange about the plot -- if you inspect browser console, it raises TypeError: attr[pt.pointNumber[0]] is undefined (this is when if (length(s) == 0 in your code).

I guess you can report it as a bug to plotly. If you need something that works now, the easiest solution is to exploit the fact that tooltip is generated correctly and add javascript code sending its content to shiny server. There you can extract variables that you need.

In the example below data is updated (ie, sent to R) when you click on the plot:

library(tidyverse) library(shiny) library(RColorBrewer) library(plotly) read_csv("https://github.com/JonMinton/housing_tenure_explorer/blob/master/data/FRS%20HBAI%20-%20tables%20v1.csv?raw=true") %>%    #read_csv("data/FRS HBAI - tables v1.csv") %>%    select(     region = regname, year = yearcode, age = age2, tenure = tenurename, n = N_ten4s, N = N_all2   ) %>%    mutate(     proportion = n / N   ) -> dta   regions <- unique(dta$region)  tenure_types <- unique(dta$tenure)   # Define UI for application that draws a histogram ui <- fluidPage(    # Application title   titlePanel("Minimal example"),    # Sidebar with a slider input for number of bins    sidebarLayout(     sidebarPanel(       sliderInput("bins",                   "Number of bins:",                   min = 1,                   max = 50,                   value = 30)     ),      # Show a plot of the generated distribution     mainPanel(       plotlyOutput("3d_surface_overlaid"),       verbatimTextOutput("selection")     )   ),    tags$script('     document.getElementById("3d_surface_overlaid").onclick = function() {         var content = document.getElementsByClassName("nums")[0].getAttribute("data-unformatted");         Shiny.onInputChange("tooltip_content", content);     };   ')  )  # Define server logic required to draw a histogram server <- function(input, output) {    output$`3d_surface_overlaid` <- renderPlotly({     # Start with a fixed example       matrixify <- function(X, colname){       tmp <- X %>%          select(year, age, !!colname)       tmp %>% spread(age, !!colname) -> tmp       years <- pull(tmp, year)       tmp <- tmp %>% select(-year)       ages <- as.numeric(names(tmp))       mtrx <- as.matrix(tmp)       return(list(ages = ages, years = years, vals = mtrx))     }       dta_ss <- dta %>%        filter(region == "UK") %>%        select(year, age, tenure, proportion)       surface_oo <- dta_ss %>%        filter(tenure == "Owner occupier") %>%        matrixify("proportion")      surface_sr <- dta_ss %>%        filter(tenure == "Social rent") %>%        matrixify("proportion")      surface_pr <- dta_ss %>%        filter(tenure == "Private rent") %>%        matrixify("proportion")      surface_rf <- dta_ss %>%        filter(tenure == "Care of/rent free") %>%        matrixify("proportion")       tooltip_oo <- surface_oo      tooltip_sr <- surface_sr      tooltip_pr <- surface_pr      tooltip_rf <- surface_rf      custom_text <- paste0(       "Year: ", rep(tooltip_oo$years, times = length(tooltip_oo$ages)), "\t",       "Age: ", rep(tooltip_oo$ages, each = length(tooltip_oo$years)), "\n",       "Composition: ",        "OO: ", round(tooltip_oo$vals, 2), "; ",       "SR: ", round(tooltip_sr$vals, 2), "; ",       "PR: ", round(tooltip_pr$vals, 2), "; ",       "Other: ", round(tooltip_rf$vals, 2)     ) %>%        matrix(length(tooltip_oo$years), length(tooltip_oo$ages))      custom_oo <- paste0(       "Owner occupation: ", 100 * round(tooltip_oo$vals, 3), " percent\n",       custom_text     ) %>%        matrix(length(tooltip_oo$years), length(tooltip_oo$ages))      custom_sr <- paste0(       "Social rented: ", 100 * round(tooltip_sr$vals, 3), " percent\n",       custom_text     ) %>%        matrix(length(tooltip_oo$years), length(tooltip_oo$ages))      custom_pr <- paste0(       "Private rented: ", 100 * round(tooltip_pr$vals, 3), " percent\n",       custom_text     ) %>%        matrix(length(tooltip_oo$years), length(tooltip_oo$ages))      custom_rf <- paste0(       "Other: ", 100 * round(tooltip_rf$vals, 3), " percent\n",       custom_text     ) %>%        matrix(length(tooltip_oo$years), length(tooltip_oo$ages))      n_years <- length(surface_oo$years)     n_ages <- length(surface_oo$ages)      plot_ly(       showscale = F     ) %>%        add_surface(         x = ~surface_oo$ages, y = ~surface_oo$years, z = surface_oo$vals,         name = "Owner Occupiers",         opacity = 0.7,         colorscale = list(           c(0,1),           c('rgb(255,255,0)' , 'rgb(255,255,0)')         ),         hoverinfo = "text",         text = custom_oo        ) %>%        add_surface(         x = ~surface_sr$ages, y = ~surface_sr$years, z = surface_sr$vals,         name = "Social renters",         opacity = 0.7,         colorscale = list(           c(0,1),           c('rgb(255,0,0)' , 'rgb(255,0,0)')         ),         hoverinfo = "text",         text = custom_sr        ) %>%        add_surface(         x = ~surface_pr$ages, y = ~surface_pr$years, z = surface_pr$vals,         name = "Private renters",         opacity = 0.7,         colorscale = list(           c(0,1),           c('rgb(0,255,0)' , 'rgb(0,255,0)')         ),         hoverinfo = "text",         text = custom_pr        ) %>%        add_surface(         x = ~surface_rf$ages, y = ~surface_rf$years, z = surface_rf$vals,         name = "Other",         opacity = 0.7,         colorscale = list(           c(0,1),           c('rgb(0,0,255)' , 'rgb(0,0,255)')         ),         hoverinfo = "text",         text = custom_rf         ) %>%        layout(         scene = list(           aspectratio = list(             x = n_ages / n_years, y = 1, z = 0.5           ),           xaxis = list(             title = "Age in years"           ),           yaxis = list(             title = "Year"           ),           zaxis = list(             title = "Proportion"           ),           showlegend = FALSE         )      )    })     output$selection <- renderPrint({     input$tooltip_content   })  }  # Run the application  shinyApp(ui = ui, server = server) 

Read More

How to show HTML text in Android while detecting clicks on specific phrases and scrolling position?

I'm getting a wall of plain HTML text from the server and need to render it in my application, but that's not all.

I also need to detect clicks on specific phrases within the text. The phrases are defined by two numbers: word count where the phrase starts and word count where it ends (e.g. from word 10 to word 15).

My intuition says that I could probably instrument the HTML with links or some JavaScript according to phrases spec and then listen for clicks on these links. However, I'm not sure how to achieve this kind of functionality in Android.

In addition, I also need to be able to programmatically observe and manipulate the scrolling position within the shown HTML text. For instance, I'll need to understand when a specific phrase is scrolled off the screen.

I guess I have three closely related questions:

1. Which View should I use to achieve the above functionality (TextView, WebView, other)?
2. How can I listen for clicks on specific parts of HTML?
3. How can I observe and manipulate the scrolling position?

Thanks

5 Answers

Answers 1

1. WebView is a quick way to do this.
2. You can map a java/kotlin function to the javascript function on the webpage to track clicks.
3. Again using simple jquery , you can achieve it.

Refer to : slymax web view

Answers 2

I think you can do this by using this method : 1.) You need to get all links from the html text you have. So to do this use this method :

 public static ArrayList<String> extractUrls(String text) {     ArrayList<String> containedUrls = new ArrayList<>();     String urlRegex = "((https?|ftp|gopher|telnet|file):((//)|(\\\\))+[\\w\\d:#@%/;$()~_?\\+-=\\\\\\.&]*)";     Pattern pattern = Pattern.compile(urlRegex, Pattern.CASE_INSENSITIVE);     Matcher urlMatcher = pattern.matcher(text);      while (urlMatcher.find()) {         containedUrls.add(text.substring(urlMatcher.start(0),                 urlMatcher.end(0)));     }     return containedUrls; } 

It will return an ArrayList of URLs, Now you need to convert the HTML data into human readable text : To do this use :

public void HtmlToString(final String data) {     Thread thread = new Thread(new Runnable() {         @Override         public void run() {             final String s = String.valueOf(Html.fromHtml(data));             runOnUiThread(new Runnable() {                 @Override                 public void run() {                     processData(s);                 }             });         }     });     thread.start(); } void processData(String s){ // Do whatever you want to do } 

We are doing this work on another thread. Now, You have text as well as links, do whatever you want with this. Now if you want to do more work on it you may do this by replacing all the links you get in array list with a special code that you can use as a placeholder like :

for(int i = 0; i < urlArray.size();i++){     yourData.replace(urlArray.get(i),"<<<YOURSPECIALCODE>>>");        } 

Now you can break your data using your Special code to get the breaks at the place of URLs. To do that :

ArrayList<String> dataArray = new ArrayList<>(yourData.split("<<<YOURSPECIALCODE>>>")); 

Now you can use these two arrays to show according to your requirements

As now You can assign different text views to different data and setOnClick Listeners to them very easily.

Hope it may help!

Thank you

Answers 3

In case you are planning to use TextView (I would prefer to if the content isn't supposed to be rendered as a full HTML, use WebView otherwise) and use Clickable span within the text content to make specific areas clickable. You can handle the click on individual span areas and perform the required actions in the click handler. You need to write multiple CustomClickableSpan classed based on the number of different clickable areas you wish to handle.

Answers 4

I guess for you're use case ideal view would be WebView.

It is easier to handle HTML in a webview rather than TextView.

TextView requires you to make use of spans for handling HTML content which is somewhat difficult when compared to handling HTML in a WebView.

And also for monitoring the scrolling behaviour, we can make use of the scroll event of an element in a webview which makes it easier to handle individual elements scroll behaviour.

So in order to make certain portions of text clickable make use of the below code snippet (Kotlin),

val htmlString = "<div>Handling the click near the specified indices</div>"         val document = Jsoup.parse(htmlString)         document.outputSettings().prettyPrint(false)         var plaintext = document.body().text()         plaintext = plaintext.replaceRange(13,18, "<a href='https://www.google.com'>${plaintext.substring(13, 18)}</a>")         document.body().html(plaintext) 

Make use of Jsoup library for parsing through the HTML content. The above code snippet will make the indexed substring clickable and point it to www.google.com. You can also add other events to it as well.

Then in order to check if an element is visible on screen you can have a javascript function like,

function isElementVisible(element)  {    var docViewTop = $(window).scrollTop();    var docViewBottom = docViewTop + $(window).height();          var elemTop = $(element).offset().top;    var elemBottom = elemTop + $(element).height();          return ((elemBottom <= docViewBottom) && (elemTop >= docViewTop));  }

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>

which takes in the element and returns true if element is visible onscreen or false if the element is not visible.

Answers 5

You can do that with SpannableString - find every link tags in your html text and set spannable wherever you need (for example for every between tag: SPANNABLE STRING).

Follow this topic, make your clickable spannable strings and show it in TextView. It's not so hard :)

Android: ClickableSpan in clickable TextView

Read More

Share Database between React Native Apps

I am building two react native apps where i need to keep few things in common like username(login) and few other information.I tried to store using AsyncStorage (React Native Storage) and everything was working but now i need to use one common data base for both the apps as the user login success in first app then the other app should also be logged in.As Asyncstorage cannot be used in this case any other option in react native.

3 Answers

Answers 1

The new privacy laws do not let share the same database for 2 applications. The only thing you can do is put the database online and access to it by both apps. At least it happen in Europe. Anyway you can't use the same AsyncStorage to 2 different apps. See more here: https://stackoverflow.com/a/48806319/8898886

Answers 2

In Android, the system design do not permit that. Every app has its own sandbox. Shortly, an app can not access the database of another app.

But it could be swindled, at your risk.

You could use a sqlite database, put the database file in a shared directory, and access that file from each app. Please note that each means really each. Every app on the device could access that file, so it is not a safe place for sensible data.

The best solution, as proposed, is to use a external db, accessible from the network, to store your data. If you belive that the effort is not worth it, you could use Firebase, for example.

Answers 3

You can not use that locally. But with Firebase you can make 2 app have same Firebase Realtime database so you can share data between 2 app (event realtime). I prefer use this react-native-firebase library for React Native https://github.com/invertase/react-native-firebase

Read More

Hierarchical queries with Mongo using $graphLookup

I have an employee collection with half a million records. Each record will have the following details.

The mongo document is as follows.

{   "_id": "234463456453643563456",   "name": "Mike",   "empId": "10",   "managerId": "8",   "projects" : [ "123", "456", "789"] } 

1. When i give any empId, it should return the complete hierarchies from that manager to the bottom level along with the following filter.

a. filter on location
b. filter on projects

The result should be like,

      10     ->>> Manager       /\      /  \     8    6  ---->> 8 & 6 reporting to manager 10     /\    /\    /  \  /  \   4    5 2   1  ---->> 4 & 5 reporting to manager 8 ... 

Any help will be appreciated for getting the hierarchical results with level?

I am not able to get the result as expected.

Sample Data :-

db.getCollection("employees").insert({"_id":"10","empId": "10","name":"Employee10","managerId":"15" });  db.getCollection("employees").insert({"_id":"8","empId": "8","name":"Employee8","managerId":"10" });  db.getCollection("employees").insert({"_id":"6","empId": "6","name":"Employee6","managerId":"10" });  db.getCollection("employees").insert({"_id":"4","empId": "4","name":"Employee4","managerId":"8" });  db.getCollection("employees").insert({"_id":"5","empId": "5","name":"Employee5","managerId":"8" });  db.getCollection("employees").insert({"_id":"2","empId": "2","name":"Employee2","managerId":"6" });  db.getCollection("employees").insert({"_id":"1","empId": "1","name":"Employee1","managerId":"6" }); 

Query :-

db.getCollection('employees').aggregate([ {     $match: {         empId : "10"     } }, {    $graphLookup: {       from: "employees",       startWith: "$empId",       connectFromField: "empId",       connectToField: "managerId",       as: "reportees",       maxDepth: 4,       depthField: "level"    } }, {    $project: {      "empId":1,      "managerId":1,      "reportees.empId":1,      "reportees.name":1,      "reportees.managerId":1,      "reportees.level":1    } } ]); 

Actual Result :-

{      "_id" : "10",      "empId" : "10",      "managerId" : "15",      "reportees" : [         {             "empId" : "1",              "name" : "Employee1",              "managerId" : "6",              "level" : NumberLong(1)         },          {             "empId" : "4",              "name" : "Employee4",              "managerId" : "8",              "level" : NumberLong(1)         },          {             "empId" : "2",              "name" : "Employee2",              "managerId" : "6",              "level" : NumberLong(1)         },          {             "empId" : "5",              "name" : "Employee5",              "managerId" : "8",              "level" : NumberLong(1)         },          {             "empId" : "6",              "name" : "Employee6",              "managerId" : "10",              "level" : NumberLong(0)         },          {             "empId" : "8",              "name" : "Employee8",              "managerId" : "10",              "level" : NumberLong(0)         }     ] } 

Expected Result :-

{      "_id" : "10",      "empId" : "10",      "managerId" : "15",      "reportees" : [         {             "empId" : "6",              "name" : "Employee6",              "managerId" : "10",              "level" : NumberLong(0),             "reportees" : [               {                "empId" : "1",                 "name" : "Employee1",                 "managerId" : "6",                 "level" : NumberLong(1)               },                {                "empId" : "2",                 "name" : "Employee2",                 "managerId" : "6",                 "level" : NumberLong(1)               }             ]         },          {             "empId" : "8",              "name" : "Employee8",              "managerId" : "10",              "level" : NumberLong(0),             "reportees" : [               {                 "empId" : "5",                  "name" : "Employee5",                  "managerId" : "8",                  "level" : NumberLong(1)               },               {                 "empId" : "4",                  "name" : "Employee4",                  "managerId" : "8",                  "level" : NumberLong(1)               }              ]         }     ] } 

Questions :-

1. Is it possible to get the expected output with $graphLookup?
2. Also, Is it possible to get the count at the top level and also for each sub level?
3. How to apply projection at all level?
4. How to apply filter on top of this?

Thanks

2 Answers

Answers 1

The official documentation on $graphLookup may provide help more or less.

https://docs.mongodb.com/manual/reference/operator/aggregation/graphLookup/

Just a kind remind.

Answers 2

That's precicsely what you would $graphLookup for (the traversal bit at least). For the filtering part you could simply use $filter or $match depending on how exactly you want to filter.

Have a look at the results of this query:

db.employees.aggregate({     $graphLookup: {       from: "pets",       startWith: "$managerIds",       connectFromField: "managerId",       connectToField: "number",       as: "managers",     } }) 

UPDATE:

In order to get the hierarchical structure that you'd like to get you could do the following. However, I wouldn't call this a pretty solution since it requires you statically define the number of levels you want to go down and also to repeat sections but it does the job for your example. Not sure, if/how easily this can be extended to more levels, either. Personally, I think a client side loop solution would be more suitable for this kind of job:

db.employees.aggregate([ {     $match: {         empId : "10"     } }, // level 0 {    $graphLookup: {       from: "employees",       startWith: "$empId",       connectFromField: "empId",       connectToField: "managerId",       as: "reportees",       maxDepth: 0    } }, {     $unwind: "$reportees" // flatten }, {     $addFields: {         "reportees.level": 0 // add level field     } }, // level 1 {    $graphLookup: {       from: "employees",       startWith: "$reportees.empId",       connectFromField: "reportees.empId",       connectToField: "managerId",       as: "reportees.reportees",       maxDepth: 0    } }, {     $group: { // group previously flattened documents back together         _id: "$_id",         empId: { $first: "$empId" },         name: { $first: "$name" },         managerId: { $first: "$managerId" },         reportees: { $push: "$reportees" },     } }, {     $addFields: {         "reportees.reportees.level": 1 // add level field     } } ]) 

UPDATE 2:

The following query gets you to where you want to be from an output structure point of view (I omitted the level field but it should be easy to add). It is, however, not particularly pretty and, again, requires you to define a maximum organisational depth upfront.

db.employees.aggregate([ {     $match: {         empId : "10"     } }, {    $graphLookup: { // get the relevant documents out of our universe of employees       from: "employees",       startWith: "$empId",       connectFromField: "empId",       connectToField: "managerId",       as: "reportees"    } }, {     $project: { // add the employee we are interested in into the array of employees we're looking at         _id: 0,         reportees: { $concatArrays: [ "$reportees", [ { _id: "$_id", empId: "$empId", name: "$name", managerId: "$managerId" } ] ] }     } }, {     $project: {         reportees: {             $let: {                 vars: {                     managers: {                         $filter: { // remove employees with no reportess so keep managers only                             input: {                                 $map: {                                     input: "$reportees",                                     as: "this",                                     in: {                                         $mergeObjects: [                                             "$$this",                                             {                                                 reportees: {                                                     $filter: { // extract reportees from list of employees                                                         input: "$reportees",                                                         as: "that",                                                         cond: {                                                             $eq: [ "$$this._id", "$$that.managerId" ]                                                         }                                                     }                                                 }                                             }                                         ]                                     }                                 }                             },                             as: "this",                             cond: { $ne: [ "$$this.reportees", [] ] }                         }                     }                 },                 in: {                     $cond: [ // this is to break the processing once we have reached a top level manager                         { $eq: [ "$$managers", [] ] },                         "$reportees",                         "$$managers"                     ]                 }             }         }     } }, // second level: exactly identical to the previous stage // third level: exactly identical to the previous stage // basically, from here onwards you would need to repeat an exact copy of the previous stage to go one level deeper ]); 

Read More

Why will changing this CSS effect using JS only work smoothly for certain values?

I am trying to move an image slowly relative to the viewport when the user scrolls the page. Similar to the effects found here https://ihatetomatoes.net/demos/parallax-scroll-effect-part-2/

If the image is moved by a small value then it moves smoothly. If it is moved by a larger amount then it becomes very janky.

var imageOffset = lastScrollY * 0.9; $image.css({top: `${imageOffset}px`});     //Runs badly  var imageOffset = lastScrollY * 0.3; $image.css({top: `${imageOffset}px`});     //Runs well 

Why does the value affect the performance so much?

I have tried all the different CSS styles (transform, top, bottom, background-position). Dev tools says that I am well in the time limit for 60fps. This happens if there is nothing but the image on the page and on multiple browsers and devices. This is also not just for images but for text or anything else as well.

Bad Version: https://jsfiddle.net/4vcg8mpk/58/

Good Version: https://jsfiddle.net/4vcg8mpk/59/

Problem most noticeable in Firefox, in Chrome it is noticeable on first scroll and then settles down. Also most noticeable using scroll wheel or trackpad instead of dragging side scroll bar

6 Answers

Answers 1

This is because you are trying to animate properties that impact page layout. These properties require the browser to recalculate layout of the DOM each time a layout property changes. You may not be experiencing performance lag in the second option because the layout repainted quickly, but that doesn't mean you are not going to eventually encounter performance issues while animating those properties.

I'd recommend checking out this article on animation performance with CSS. It is old, but the information is still valid. I know you say that you tried animating other properties, but I would recommend taking a look through all of those recommendations and then implementing something that is going to be "cheap" for the browser.

Answers 2

You are using a scroll linked effect, which are janky in modern browsers because the scroll event lags behind what the users sees. Explained here:

Often scrolling effects are implemented by listening for the scroll event and then updating elements on the page in some way (usually the CSS position or transform property.) You can find a sampling of such effects at CSS Scroll API: Use Cases.

These effects work well in browsers where the scrolling is done synchronously on the browser's main thread. However, most browsers now support some sort of asynchronous scrolling in order to provide a consistent 60 frames per second experience to the user. In the asynchronous scrolling model, the visual scroll position is updated in the compositor thread and is visible to the user before the scroll event is updated in the DOM and fired on the main thread. This means that the effects implemented will lag a little bit behind what the user sees the scroll position to be. This can cause the effect to be laggy, janky, or jittery — in short, something we want to avoid.

Source: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Scroll-linked_effects

You will have of course the least jankiest experience when there is less of a difference between what the user immediately sees and what is going to change after the scroll-event has executed and changed the CSS top-property.

If you multiply the top property by a number more closer to 1 (or even more) then there will be a bigger difference between what the user sees immediately and what the user sees later on (when the scroll-event has done its thing).

If you multiply the top property by a number more closer to 0 then there will be less of a difference between the immediate experience and what changes later on (when the scroll-event has done its thing).

Since there is a big difference between multiplying by 0.3 and multiplying by 0.95, the jankiness associated with asynchronous scrolling will also become more apparent, which should answer your question:

Why does the value affect the performance so much?

Answers 3

The issue could be resolved with position: fixed: https://jsfiddle.net/4vcg8mpk/62/

// required code block #contextImage {     position: fixed; } 

---

The issue is that scroll event is triggered AFTER scrolling is done, this means that scroll event is not cancelable, and also that browser might render previous DOM before JS executed...

The performance difference you mentioned is not noticeable in my environment, but according to my previous experience, browser could ignore DOM updates during scroll, if JS execution(maybe +layouting +painting) is not finished before next screen should be painted.

So browser needs to display next frame(for smooth scrolling UX) but DOM updates are not yet processed. It just uses cached version, then triggers next 'scroll' event, then trying to paint next frame, repeat...

---

to create smooth scrolling experience - the system must update positions of all elements in single animation frame, so it cannot do updates during 'scroll' event.

Answers 4

All you have to do, is but the course. Or, you can just get the trial.

Or, just look here:

Req. HTML:

<div id="someText">    Lorem ipsum dolor sit amet, consectetur adipiscing elit. Praesent eget turpis in nulla ultricies pretium. Nullam quis molestie velit. Vestibulum varius iaculis risus, sit amet gravida nulla efficitur quis. Sed fringilla congue nunc id tincidunt. Nam eget nunc quis est accumsan tincidunt. Aliquam maximus, nunc nec facilisis malesuada, nisi neque pulvinar magna, id hendrerit nibh nisi in justo. Morbi consequat massa massa, sed dictum mi dignissim et. Etiam id ullamcorper ante. Etiam ac magna id libero varius sollicitudin. Nulla varius blandit tristique. Mauris finibus gravida felis, at interdum eros ultricies in. Suspendisse vestibulum ornare interdum. Mauris venenatis vel nisl et efficitur. Nulla tristique nibh vel felis tincidunt egestas et eu justo.  Praesent congue ex id tempus interdum. Nunc placerat sollicitudin enim nec volutpat. Aenean nec dignissim turpis. Sed ut orci lobortis, consequat ante eget, posuere diam. Vestibulum ac sagittis nulla. Nam consequat ante nisl, at dapibus nibh ultrices at. Etiam ut elit feugiat, pretium augue sit amet, rutrum est. Sed et augue sit amet ligula mattis posuere. Nullam sed commodo nulla.  Vestibulum hendrerit felis risus, a consectetur dui sagittis at. Morbi in accumsan dolor. Mauris sodales consectetur tortor, in maximus tellus efficitur eget. Nullam posuere hendrerit arcu, id fringilla lectus ultricies sit amet. Integer sit amet dignissim libero, commodo mollis massa. Sed est nibh, mollis quis orci nec, egestas tempus leo. Etiam quis pretium mi. Donec in bibendum purus. Curabitur accumsan erat felis, ut lobortis diam luctus a. Praesent non arcu vitae tortor blandit tempus.  Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Vestibulum ac enim vitae augue gravida pretium a et nibh. Vivamus pretium turpis quis orci venenatis efficitur. Aliquam ac pulvinar turpis, in facilisis erat. Pellentesque consectetur sodales finibus. Proin gravida, erat id pharetra condimentum, erat nisl cursus tellus, vitae viverra est orci non nunc. Nulla facilisi. Proin quis porta dolor. Suspendisse aliquam at nibh vitae pharetra. Vivamus a facilisis mi, ac faucibus nunc. Ut pharetra, diam convallis suscipit vulputate, urna quam tincidunt leo, in suscipit neque nibh vel nisl. Nullam suscipit lorem eget venenatis luctus. Sed purus dui, dignissim eget sapien quis, posuere efficitur magna. Donec quis lectus vitae erat vehicula hendrerit vel in erat.  Vestibulum sollicitudin consectetur enim. Donec lectus mi, vulputate eu tempor quis, porttitor non lacus. Nulla nisl risus, lacinia vitae magna et, hendrerit porta turpis. Sed scelerisque quam non porttitor laoreet. Sed nec ipsum metus. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Aenean accumsan bibendum rutrum. Nullam interdum elit auctor augue mattis dapibus. </div>  <div id="contextImage">  </div> 

Req. CSS:

someText {   height: 300vh;   width: 200px;   position: absolute; }  contextImage {   will-change: top;     transform: translateZ(0);   position: fixed;   height: 80vh;   width: 100%;   background-image: url(https://www.gettyimages.ca/gi-resources/images/Homepage/Hero/UK/CMS_Creative_164657191_Kingfisher.jpg);   background-size: cover;  } 

Req. JavaScript & JQuery:

var $window = $(window); var windowHeight = $window.height();  var $exampleDiv = $('#someText'); var $contextImage = $('#contextImage');  //Variables for debouncing scroll  var lastScrollY = window.pageYOffset;    var ticking = false;  //Debouncing scroll events window.addEventListener('scroll', getYOffset, false);  //Keeps track of last sccroll event function getYOffset() {   lastScrollY = window.pageYOffset;   requestTick();         } //stops further rAFs until current is complete function requestTick() {     if (!ticking) {         requestAnimationFrame(parallax);         ticking = false;     } }  //When new animation frame availible then do animation function parallax() {     if (lastScrollY < (4 * windowHeight)) {         var imageOffset = (-lastScrollY * 0.1);         $contextImage.css({transform: `translateY(${imageOffset}px)`});     }      // Allow new rAFs     ticking = false;  } 

There you go. Problem solved.

Hope this helps!!!

Answers 5

Something like this?

#contextImage {      background-image: url("https://www.gettyimages.ca/gi-resources/images/Homepage/Hero/UK/CMS_Creative_164657191_Kingfisher.jpg");      min-height: 500px;       background-attachment: fixed;      background-position: center;      background-repeat: no-repeat;      background-size: cover;  }

<!DOCTYPE html>  <html>  <head>  <meta name="viewport" content="width=device-width, initial-scale=1">  </head>  <body>      <div id="contextImage"></div>    <div id="someText">     Lorem ipsum dolor sit amet, consectetur adipiscing elit. Praesent eget turpis in nulla ultricies pretium. Nullam quis molestie velit. Vestibulum varius iaculis risus, sit amet gravida nulla efficitur quis. Sed fringilla congue nunc id tincidunt. Nam eget nunc quis est accumsan tincidunt. Aliquam maximus, nunc nec facilisis malesuada, nisi neque pulvinar magna, id hendrerit nibh nisi in justo. Morbi consequat massa massa, sed dictum mi dignissim et. Etiam id ullamcorper ante. Etiam ac magna id libero varius sollicitudin. Nulla varius blandit tristique. Mauris finibus gravida felis, at interdum eros ultricies in. Suspendisse vestibulum ornare interdum. Mauris venenatis vel nisl et efficitur. Nulla tristique nibh vel felis tincidunt egestas et eu justo.    Praesent congue ex id tempus interdum. Nunc  gplacerat sollicitudin enim nec volutpat. Aenean nec dignissim turpis. Sed ut orci lobortis, consequat ante eget, posuere diam. Vestibulum ac sagittis nulla. Nam consequat ante nisl, at dapibus nibh ultrices at. Etiam ut elit feugiat, pretium augue sit amet, rutrum est. Sed et augue sit amet ligula mattis posuere. Nullam sed commodo nulla.    Vestibulum hendrerit felis risus, a consectetur dui sagittis at. Morbi in accumsan dolor. Mauris sodales consectetur tortor, in maximus tellus efficitur eget. Nullam posuere hendrerit arcu, id fringilla lectus ultricies sit amet. Integer sit amet dignissim libero, commodo mollis massa. Sed est nibh, mollis quis orci nec, egestas tempus leo. Etiam quis pretium mi. Donec in bibendum purus. Curabitur accumsan erat felis, ut lobortis diam luctus a. Praesent non arcu vitae tortor blandit tempus.  Praesent congue ex id tempus interdum. Nunc  gplacerat sollicitudin enim nec volutpat. Aenean nec dignissim turpis. Sed ut orci lobortis, consequat ante eget, posuere diam. Vestibulum ac sagittis nulla. Nam consequat ante nisl, at dapibus nibh ultrices at. Etiam ut elit feugiat, pretium augue sit amet, rutrum est. Sed et augue sit amet ligula mattis posuere. Nullam sed commodo nulla.    Vestibulum hendrerit felis risus, a consectetur dui sagittis at. Morbi in accumsan dolor. Mauris sodales consectetur tortor, in maximus tellus efficitur eget. Nullam posuere hendrerit arcu, id fringilla lectus ultricies sit amet. Integer sit amet dignissim libero, commodo mollis massa. Sed est nibh, mollis quis orci nec, egestas tempus leo. Etiam quis pretium mi. Donec in bibendum purus. Curabitur accumsan erat felis, ut lobortis diam luctus a. Praesent non arcu vitae tortor blandit tempus.  Praesent congue ex id tempus interdum. Nunc  gplacerat sollicitudin enim nec volutpat. Aenean nec dignissim turpis. Sed ut orci lobortis, consequat ante eget, posuere diam. Vestibulum ac sagittis nulla. Nam consequat ante nisl, at dapibus nibh ultrices at. Etiam ut elit feugiat, pretium augue sit amet, rutrum est. Sed et augue sit amet ligula mattis posuere. Nullam sed commodo nulla.    Vestibulum hendrerit felis risus, a consectetur dui sagittis at. Morbi in accumsan dolor. Mauris sodales consectetur tortor, in maximus tellus efficitur eget. Nullam posuere hendrerit arcu, id fringilla lectus ultricies sit amet. Integer sit amet dignissim libero, commodo mollis massa. Sed est nibh, mollis quis orci nec, egestas tempus leo. Etiam quis pretium mi. Donec in bibendum purus. Curabitur accumsan erat felis, ut lobortis diam luctus a. Praesent non arcu vitae tortor blandit tempus.    Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Vestibulum ac enim vitae augue gravida pretium a et nibh. Vivamus pretium turpis quis orci venenatis efficitur. Aliquam ac pulvinar turpis, in facilisis erat. Pellentesque consectetur sodales finibus. Proin gravida, erat id pharetra condimentum, erat nisl cursus tellus, vitae viverra est orci non nunc. Nulla facilisi. Proin quis porta dolor. Suspendisse aliquam at nibh vitae pharetra. Vivamus a facilisis mi, ac faucibus nunc. Ut pharetra, diam convallis suscipit vulputate, urna quam tincidunt leo, in suscipit neque nibh vel nisl. Nullam suscipit lorem eget venenatis luctus. Sed purus dui, dignissim eget sapien quis, posuere efficitur magna. Donec quis lectus vitae erat vehicula hendrerit vel in erat.    Vestibulum sollicitudin consectetur enim. Donec lectus mi, vulputate eu tempor quis, porttitor non lacus. Nulla nisl risus, lacinia vitae magna et, hendrerit porta turpis. Sed scelerisque quam non porttitor laoreet. Sed nec ipsum metus. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Aenean accumsan bibendum rutrum. Nullam interdum elit auctor augue mattis dapibus.  </div>  </body>  </html>

Answers 6

In your example, the smooth of your transition is not depend on the value. Both version (big and small move) are all weird. But with the small move, it's hard to see the difference, so you may feel it smoothly. The real reason of the problem is requestAnimationFrame.

In theory, requestAnimationFrame is good for animation but in this case, it's bad. Let's remove the requestAnimationFrame and use your function directly. Now it's work smoothly in both chrome and firefox and with any value. Here is the fiddle https://jsfiddle.net/4vcg8mpk/64/.

requestAnimationFrame means you want to run your function in next frame. So, when you scrolling down the body, the browser does scroll down (it makes your image go up) and excutes your function in next frame (it makes your image go down). Seem like the frame rate is not high enough so your image will up down up down. It makes your transition janky and it's easier to see with lagre move.

When remove requestAnimationFrame your function will excute as soon as the scroll happen and run with every scroll action so it will be truly smooth.

Read More

Exclude the overlay app icon from a screenshot

Im just brainstorming an idea and Im checking its possibility before starting the code. Lets say I have an app that takes a screenshot whenever I tap on its overlayed icon on screen,is there a way to exclude my app's icon from the resulting image?

2 Answers

Answers 1

The media projection APIs simply capture what is on the screen. There is no means to say "ignore this window", other than by making the window not be there (or be transparent) at the time of the screen capture.

Answers 2

very simple

hide the icon at the capture time and then make it visible after that.

Read More

If I have two content types with a relationship between them in Drupal 8, why would I want to put a reference field in both content types?

If I have two content types with a relationship between them in Drupal 8, in order to represent the relationship, I can put a reference field to the other content type in one of these two content types.

But in this video tutorial, it shows an example where both content types contains a reference field to each other.

In what situations would I want to put a reference field in both content types?

1 Answers

Answers 1

If you are user of the example website in the video, I can think of these two use cases:

• I am viewing the Event Detail Page, then I want to know who is sponsoring the event.

• I am viewing the User Group Detail Page, then I want to know what events the group is sponsoring.

Then you would have to discuss the implementation.

In the video, they go for two reference fields, I see it as a shortcut so that you have your Event Detail Page and User Group Detail Page ready. But then you have a new problem, you will have to keep both fields in sync. Having this type of redundancy wouldn't be considered a good practice in database design.

Alternatively, if you don't want to deal with field synchronization, you only need a reference field. Let's say the Event will have the reference field, now we would have to come up with a way to query the events of a group for the User Group Detail Page. A Drupal View would be the most common way to do this.

I think the instructor in the video doesn't want to introduce views at this point of the training, because it is a broad topic, then probably he will go back and revisit the implementation later in the course.

Read More

Long running script from flask endpoint

I've been pulling my hair out trying to figure this one out, hoping someone else has already encountered this and knows how to solve it :)

I'm trying to build a very simple Flask endpoint that just needs to call a long running, blocking php script (think while true {...}). I've tried a few different methods to async launch the script, but the problem is my browser never actually receives the response back, even though the code for generating the response after running the script is executed.

I've tried using both multiprocessing and threading, neither seem to work:

# multiprocessing attempt @app.route('/endpoint') def endpoint():   def worker():     subprocess.Popen('nohup php script.php &', shell=True, preexec_fn=os.setpgrp)    p = multiprocessing.Process(target=worker)   print '111111'   p.start()   print '222222'   return json.dumps({     'success': True   })  # threading attempt @app.route('/endpoint') def endpoint():   def thread_func():     subprocess.Popen('nohup php script.php &', shell=True, preexec_fn=os.setpgrp)    t = threading.Thread(target=thread_func)   print '111111'   t.start()   print '222222'   return json.dumps({     'success': True   }) 

In both scenarios I see the 111111 and 222222, yet my browser still hangs on the response from the endpoint. I've tried p.daemon = True for the process, as well as p.terminate() but no luck. I had hoped launching a script with nohup in a different shell and separate processs/thread would just work, but somehow Flask or uWSGI is impacted by it.

Update

Since this does work locally on my Mac when I start my Flask app directly with python app.py and hit it directly without going through my Nginx proxy and uWSGI, I'm starting to believe it may not be the code itself that is having issues. And because my Nginx just forwards the request to uWSGI, I believe it may possibly be something there that's causing it.

Here is my ini configuration for the domain for uWSGI, which I'm running in emperor mode:

[uwsgi] protocol = uwsgi max-requests = 5000 chmod-socket = 660 master = True vacuum = True enable-threads = True auto-procname = True procname-prefix = michael- chdir = /srv/www/mysite.com module = app callable = app socket = /tmp/mysite.com.sock 

3 Answers

Answers 1

This kind of stuff is the actual and probably main use case for Python Celery (http://www.celeryproject.org/). As a general rule, do not run long running jobs that are CPU-bound in the wsgi process. It's tricky, it's inefficient, and most important thing, it's more complicated than setting up an async task in a celery worker. If you want to just prototype you can set the broker to memory and not using an external server, or run a single threaded redis on the very same machine.

This way you can launch the task, call task.result() which is blocking, but it blocks in an IO-bound fashion, or even better you can just return immediately by retrieving the task_id and build a second endpoint /result?task_id=<task_id> that checks if result is available:

result = AsyncResult(task_id, app=app) if result.state == "SUCCESS":    return result.get() else:    return result.state  # or do something else depending on the state 

This way you have a non-blocking wsgi app that does what is best suited for: short time CPU-unbound calls that have IO calls at most with OS-level scheduling, then you can rely directly to the wsgi server workers|processes|threads or whatever you need to scale the API in whatever wsgi-server like uwsgi, gunicorn, etc. for the 99% of workloads as celery scales horizontally by increasing the number of worker processes.

Answers 2

This approach works for me, it calls the timeout command (sleep 10s) in the command line and lets it work in the background. It returns the response immediately.

@app.route('/endpoint1') def endpoint1():     subprocess.Popen('timeout 10', shell=True)     return 'success1' 

However, not testing on WSGI server, but just locally.

Answers 3

Since this works locally, but doesn't with NGINX and uWSGI ...

• Have you tried adding processes = 2 and threads = 2 to your uWSGI config?
• Have you tried running this without NGINX?

Read More

How does the YouTube android app select video codec?

I have enabled stats for nerds option in YouTube android app and played the same video in "Vivo V9" and "Nexus 5" device.

Vivo V9 : It played the video in WebM format which is basically "VP8 or VP9" codec.

Nexus 5 : It played the video in MP4 format which is basically "H264 or H265" codec.

So, based on the device YouTube app selects video codec.

Question : How does it do ? I know internally it uses ExoPlayer for playing video but ExoPlayer by default doesn't give functionality.

1 Answers

Answers 1

Different codecs may require different licences, which can cost much. Moreover codecs can be both software and hardware. There is no problem with many SW codecs but as name states HW codecs require specific chip which also increases cost and occupy space. That's why there is big variety between device manufactures and even between two devices from the same manufacturer. They just simply want to cut costs.

It's very common that one device has only a subset of most popular codecs and even if it has let's say both VP8 and H264 then one of them can be HW codec while other can be SW codec in which case usually HW codec will be preferred. YouTube (here I mean youtube.com not YouTube app) serves videos in different formats, so the device can choose optimal codec for it's capabilities.

Now as to choosing the right codec YouTube app can use MediaCodec API from Android. Please check e.g. this and / or it even can provide it's own SW codecs, so I would say that the behavior is platform dependent.

Last things are corner cases e.g. something can be played / recorded in background e.g. camera is turned on and screen recording happens while using YouTube app. Again it depends on device but HW codecs have limitations to the number of media codec instances and in such corner case it's possible that even if device has some HW codec YouTube app may be forced to choosing SW codec.

Read More

creating and appending to a list in SQLAlchemy database table

I am learning SQLAlchemy and I am stuck. I have a SQL table (table1) has two fields: 'name' and 'other_names'

I have an excel file with two columns:

first_name alias    paul   patrick john   joe simon  simone john   joey john   jo 

I want to read the excel file into my table1, so that it looks like this (i.e. all of the aliases for the same line are on one row):

paul    patrick john    joe,joey,jo simon   simone 

This is the idea that I was trying to do. The code (with comments) that I tried:

for line in open('file.txt', 'r'): #for each line in the excel file         line = line.strip().split('\t') #split each line with a name and alias         first_name = line[0] #first name is the name before the tab         alias = line[1] #alias is the name after the tab         instance =          Session.query(session,tbs['table1'].name).filter_by(name=first_name) #look through the database table, by name field, and see if the first name is there          list_instance = [x[0] for x in instance] #make a list of first names already in database table         if first_name not in list_instance: #if the excel first name is not in the database table               alias_list = [] #make an empty list               alias_list.append(alias) #append the alias               name_obj = lib.get_or_create( #small function to make db object               session,               tbs["table1"],               name = first_name, #add first name to the name field               other_names = alias_list # add alias list to the other_names field             )          elif first_name in list_instance: #elif first name already in db              alias_list.append(alias) #append the alias to the alias list made above              name_obj = lib.get_or_create(              session,              tbs["table1"],              name = first_name,              other_names = alias_list #create object as before, but use updated alias list     ) 

The problem is that I can get the above code to run with no errors, but also the output is not an appended list, it is simply a database table that looks like the excel file; i.e.

name   alias paul   patrick john   joe simon  simone john   joey john   jo 

Could someone point out where I am going wrong, specifically, how do i amend this code? Please let me know if the question is unclear, I've tried to make it a simple example. Specifically, how do I initialise and add to lists as a field entry in a SQLalchemy db table.

Update 1: I have updated my code according to kind suggestion below. However I still have the issue. This is the full aim, code and test file: The aim:

I have a table in the database (see below for test file going into table).The table has two fields, name (the latin name e.g. homo sapiens) and other names (the common names e.g. human, man). I want to update a field (other names) in the table, so instead of having:

Rana rugosa human    Rana rugosa man  Rana rugosa frog     Rana rugosa cow 

I have:

Rana rugosa human,man,frog,cow 

The test_data file looks like this:

origin_organism        common_name         tested_organism Rana rugosa            human                - Rana rugosa            man                  - Rana rugosa            frog                 homo sapiens Rana rugosa            cow                  Rana rugosa Rana rugosa            frog                 Rana rugosa Rana rugosa            frog                 - Rana rugosa            frog                 - Rana rugosa            frog                homo sapiens -                      -                   - -                      -                   homo sapiens -                      -                   - -                      -                   - -                      -                   - -                      -                   - streptococcus pneumoniae    -              - 

The code:

import sys  from sqlalchemy.orm  import *  from sqlalchemy  import *  from dbn.sqlalchemy_module  import lib  import pd  engine = lib.get_engine(user="user", psw="pwd", db="db", db_host="111.111.111.11") Base = lib.get_automapped_base(engine) session = Session(engine) tbs = lib.get_mapped_classes(Base) session.rollback() df = pd.read_excel('test_data.xlsx', sheet_name = 'test2')     for index, row in df.iterrows():       origin_latin_name = row['origin_organism'].strip().lower()     other_names_name = row['common_name'].strip().lower()     tested_species = row['tested_organism'].strip().lower()   if origin_latin_name not in [None, "None", "", "-"]:     instance = [x[0] for x in Session.query(session,tbs['species'].name).filter_by(name=origin_latin_name).all()]     if origin_latin_name not in instance:         origin_species = lib.get_or_create(             session,             tbs["species"],             name = origin_latin_name,             other_names = other_names_name         )      elif origin_latin_name in instance:         other_names_query = Session.query(session,tbs['species'].other_names).filter_by(name=origin_latin_name)         other_names_query_list = [x for x in other_names_query]         original_list2 = list(set([y for y in x[0].split(',') for x in other_names_query_list]))         if other_names_name not in original_list2:             original_list2.append(other_names_name)             new_list = ','.join(original_list2)             new_names = {'other_names':','.join(original_list2)}          origin_species = lib.get_or_create(             session,             tbs["species"],             name = origin_latin_name,             other_names = new_list         ) 

The part from the elif statement doesn't work. I've ran into two problems:

(1) The most recent error I got: NameError: name 'new_list' is not defined

(2) another error I got is that I have another table further on

map1 = lib.get_or_create(     session,     tbs["map1"],     age_id_id = age,     name_id_id = origin_species.id     ) 

...and it said that origin_species cannot be found, but I think this is linked to the elif statement, that somehow the origin_species object is not being updated properly.

If anyone could help I would appreciate it.

1 Answers

Answers 1

Simple mistake. You aren't giving it a list. I'm not sure why they end up in different rows, however, I would change the following because at the moment I don't see where you split the names into a list, all I see is you assigning a string onto a list using append.

alias_list = alias.split(',') 

Which could also be:

alias_list = line[1].split(',') 

Output:

alias_list:    ['Name1','Name2','Name3'] 

Currently your code outputs:

alias_list = ['Name1,Name2,Name3'] 

Which, whilst it is technically a list by data type, it is a worthless list for the way you want to use it. This is because alias_list[0] would return the entire string, as opposed to 'Name1'

WORD OF WARNING:

Your code is creating a list unnecessarily. You don't need a list in your database, you can easily achieve what you wabt by using the string that is evaluated when you read the excel file.

What you should do IMHO is to store the string of names as a whole string, then if you need to query the aliases of someone, then you can split the string on the other side, if that makes sense?

Read More

How to show uploaded php file as plain text instead of executing it in wordpress?

Edit a testme.php file in /tmp.

<?php echo  "test"; ?> 

Edit a new post titled test and upload file /tmp/testme.php ,pubish it with url http://home.local/wp/?p=4785.

I want to see the content in testme,click it,pop up new window in wordpress.

Go on to click it.test shown in webpage.

My expect :
1.just click testme in test post for one time.
2.show the testme.php as plain text ,

<?php echo  "test"; ?> 

instead of the result of executing testme.php.

test 

I make a configuration according some material show php file as plain text in apache.

sudo vim /etc/apache2/sites-available/000-default.conf

<VirtualHost *:80>     ServerName www.home.local     ServerAdmin webmaster@localhost     DocumentRoot /var/www/html     ErrorLog ${APACHE_LOG_DIR}/error.log     CustomLog ${APACHE_LOG_DIR}/access.log combined         <Directory /var/www/html>             Options Indexes FollowSymLinks MultiViews             AllowOverride All             allow from all             php_flag engine off             AddType text/plain php         </Directory> </VirtualHost> 

Reboot apache2(build in debian).

sudo systemctl restart apache2 

To open the post http://home.local/wp/?p=4785,i got the following output in webpage:

<?php /**  * Front to the WordPress application. This file doesn't do anything, but loads  * wp-blog-header.php which does and tells WordPress to load the theme.  *  * @package WordPress  */  /**  * Tells WordPress to load the WordPress theme and output it.  *  * @var bool  */ define('WP_USE_THEMES', true);  /** Loads the WordPress Environment and Template */ require( dirname( __FILE__ ) . '/wp-blog-header.php' ); 

2 Answers

Answers 1

You already have the right code — AddType text/plain php, which will make Apache treats PHP files (or files where the name ends with .php) as plain-text files.

But assuming the following:

• You have WordPress installed in the /var/www/html directory.

• The PHP files are uploaded to the default uploads folder in WordPress (wp-content/uploads).

If you set the directory to /var/www/html/wp-content/uploads as in:

<Directory /var/www/html/wp-content/uploads>   AddType text/plain php </Directory> 

You'd get the results you wanted — only PHP files in the uploads folder will be treated as plain-text files, and not all PHP files in the /var/www/html directory. This explains the issue with "To open the post http://home.local/wp/?p=4785, I got the following output", where that output is the code in the file /var/www/html/index.php. I mean, you used <Directory /var/www/html>, which makes Apache treats the index.php file as a plain-text file, instead of executing the PHP code in the file.

ALTERNATE METHOD: Use the .htaccess file.

Particularly if you can't edit or have no access to the Apache's configuration (.conf) file.

1. Create .htaccess in the uploads folder, if it's not already there.

2. Then add AddType text/plain php in that file.

Additional Notes

I want to see the content in testme.php, click it, pop up new window

I'm sure you can do that or already have the code/solution, but an easy way, is just add target="_blank" to the attachment/file link.. or with JavaScript, you can use window.open().

And you must take full security measures since allowing people to upload PHP files could harm your site and/or your site users.

Answers 2

What are you trying to accomplish? It seems you're trying to get a hyperlink that displays the contents of a PHP file. WordPress posts are stored in the database. So when you say "publish it", what are you talking about? The database entry that refers to a WordPress post, or the PHP file? If all you want on the screen is the unexecuted contents of the PHP file, you shouldn't be publishing a WordPress "post". What is contained in the post? If you're talking about putting a PHP file as the CONTENT of a post, that would be a different thing too. Probably the easiest way to display the content of a PHP file is to just rename it to a text file.

myFile.php => myFile.php.txt

Read More

rxjava + retrofit - How to wait and get the result of first observable in android?

I've recently started learning retrofit and rxjava. I'm looking for any ideas on how to wait ang get the result of first observable. Basically, I want to apply it on a simple login. The first api call is getting the server time. The second api call will wait the result of the first call (which is the server time) and utilize it.

                Retrofit retrofit = RetrofitClient.getRetrofitClient();                 LocalServerInterface apiInterface = retrofit.create(LocalServerInterface .class);                  Observable<ServerTime> timeObservable = retrofit                         .create(LocalServerInterface .class)                         .getTime()                         .subscribeOn(Schedulers.newThread())                         .observeOn(AndroidSchedulers.mainThread());                  Observable<ServerNetwork> serverNetworkObservable = retrofit                         .create(LocalServerInterface .class)                         .getNetworks(//valuefromapicall1, anothervalue)                         .subscribeOn(Schedulers.newThread())                         .observeOn(AndroidSchedulers.mainThread()); 

Now, I'm stuck right here. On second observable, particularly on getNetworks method, I wanted to use what I got from first observable. Any ideas?

EDIT:

I wanted to process first the result of call 1 before supplying it to the api call 2. Is it possible?

1 Answers

Answers 1

First, do not recreate LocalServerInterface each time, create one and reuse it. Creation of the instance of the interface is an expensive operation.

LocalServerInterface apiInterface = retrofit.create(LocalServerInterface.class) 

And to make the second observable start with the result of the first observable, you need to do flatMap.

Observable<ServerNetwork> serverNetworkObservable = apiInterface                         .getTime()                         .flatMap(time -> apiInterface.getNetworks(time, anothervalue))                         .subscribeOn(Schedulers.newThread())                         .observeOn(AndroidSchedulers.mainThread()); 

See the flatMap documentation for more info.

IMPORTANT NOTE. In this case, when only one response will be emitted by the first observable, there's no difference between using flatMap and concatMap. For the other cases, consider the difference between flatMap and concatMap.

Read More

Unable to create a loop to compare the content of two sheets

I've written a script which is supposed to compare the content of column A between two sheets in a workbook to find out if there are partial matches. To be clearer: If any of the content of any cell in coulmn A in sheet 1 matches any of the content of any cell in coulmn A in sheet 2 then that will be a match and the script will print that in immediate window.

This is my attempt so far:

Sub GetPartialMatch()     Dim paramlist As Range      Set paramlist = Sheets(1).Range("A2:A" & Cells(Rows.Count, 1).End(xlUp).Row)      For Each cel In Sheets(2).Range("A2:A" & Cells(Rows.Count, 1).End(xlUp).Row)         If InStr(1, cel(1, 1), paramlist, 1) > 0 Then  'I used "paramlist" here as a placeholder as I can't use it            Debug.Print cel(1, 1)         End If     Next cel End Sub 

The thing is I can't make use of this paramlist defined within my script. I just used it there as a placeholder.

5 Answers

Answers 1

You want a double loop.

Sub GetPartialMatch()     Dim paramlist As Range     Dim cel as Range, cel2 as Range ; declare all variables!      Set paramlist = Sheets(1).Range("A2:A" & Cells(Rows.Count, 1).End(xlUp).Row)      For Each cel In Sheets(2).Range("A2:A" & Cells(Rows.Count, 1).End(xlUp).Row)         For Each cel2 in paramlist 'Sheets(1).Range("A2:A" & Cells(Rows.Count, 1).End(xlUp).Row)             If InStr(1, cel(1, 1), cel2, 1) > 0 Then                   Debug.Print cel(1, 1)             End If         Next cel2     Next cel End Sub 

Always use Option Explicit. Always.

This may be easier using a helper column and a formula, where the row in the helper column indicates TRUE if a MATCH is found. No VBA then. And it will be inherently faster.

Answers 2

a very fast approach is given by the use of arrays and Application.Match() function:

Sub GetPartialMatch()     Dim paramlist1 As Variant, paramlist2 As Variant     Dim cel As Range     Dim i As Long      paramlist1 = Sheets(1).Range("A2", Sheets(1).Cells(Rows.Count, 1).End(xlUp)).Value ' collect all sheets(1) column A values in an array     paramlist2 = Sheets(2).Range("A2", Sheets(2).Cells(Rows.Count, 1).End(xlUp)).Value ' collect all sheets(2) column A values in an array      For i = 1 To UBound(paramlist1) ' loop through paramlist1 array row index         If Not IsError(Application.Match(paramlist1(i, 1), paramlist2, 1)) Then Debug.Print paramlist1(i, 1) ' if partial match between current paramlist1 value and any paramlist2 value, then print it     Next End Sub 

if you want an exact match just use 0 as the last parameter in Match() function, i.e.:

If Not IsError(Application.Match(paramlist1(i, 1), paramlist2, 0)) Then Debug.Print paramlist1(i, 1) ' if exact match between current paramlist1 value and any paramlist2 value, then print it 

BTW, if you need an exact match you could also use Autofilter() method of Range object with xlFilterValues as its Operator parameter:

Sub GetPartialMatch2()     Dim paramlist As Variant     Dim cel As Range      paramlist = Application.Transpose(Sheets(1).Range("A2", Sheets(1).Cells(Rows.Count, 1).End(xlUp)).Value) ' collect all sheets(1) column A values in an array      With Sheets(2).Range("A1", Sheets(2).Cells(Rows.Count, 1).End(xlUp)) ' reference sheets(2) column A cells from row 1 (header) down to last not empty one         .AutoFilter field:=1, Criteria1:=paramlist, Operator:=xlFilterValues ' filter referenced range with 'paramlist'         If Application.WorksheetFunction.Subtotal(103, .Cells) > 1 Then ' if any filtered cell other then header             For Each cel In .Offset(1).Resize(.Rows.Count - 1).SpecialCells(xlCellTypeVisible) ' loop through all sheets(2) filtered cells but the header                 Debug.Print cel.Value2             Next         End If         .Parent.AutoFilterMode = False 'remove filter     End With End Sub 

Answers 3

Have you tried adding in:

Application.Screenupdating = false Application.Calculation = xlCalculationManual  ...Code...  Application.Screenupdating = true Application.Calculation = xlCalculationAutomatic 

These turn off the screen updating and automatic calculation of formulas within your instance of excel which can help speed up code a lot, you just have to remember to turn them back on at the end or you might give yourself a bit of a headache. It should be noted, though, that if you turn off screenupdating you won't be able to see the results roll in. You'll have to scroll backwards at the end

Another thing to consider would be store the data in an array before hand and doing the operations to the array and simply pasting it back in to the sheet. Accessing the sheet excessively slows down code drastically. Working with the accepted answer provided by @AJD, I made a few changes that will hopefully speed it up.

Sub macro()  Dim paramlist() As Variant Dim DataTable() As Variant Dim cell1 As Variant Dim cell2 As Variant  paramlist() = Sheets(1).Range("A2:A" & Worksheets(1).Cells(Rows.Count, 1).End(xlUp).Row).Value DataTable() = Sheets(2).Range("A2:A" & Worksheets(2).Cells(Rows.Count, 1).End(xlUp).Row).Value   For Each cell1 In paramlist     For Each cell2 In DataTable         If InStr(1, cell2, cell1, 1) > 0 Then             Debug.Print cell1             exit for         End If     Next cell2 Next cell1  End Sub 

I would have suggested this under the accepted answer as a suggestion, but unfortunately, I don't have enough rep to comment yet.

Edit: switching the order of the for loops allows you to insert a more efficient exit for and can allow you to skip large portions of data within the search array

Answers 4

Not sure if this is any faster (it uses pretty much the same algorithm, a loop inside of a loop), but I would argue it's a bit clearer:

Sub SearchForPartialMatches()     Dim needle1 As Range, needle2 As Range      Set needle1 = Excel.Worksheets(1).Range("$B$2")      Do While needle1.Value <> ""         Set needle2 = Excel.Worksheets(2).Range("$B$2")          Do While needle2.Value <> ""             If InStr(1, needle1.Value, needle2.Value) > 0 Then                 Debug.Print needle1.Value, needle2.Value             End If             Set needle2 = needle2.Offset(rowoffset:=1)         Loop         Set needle1 = needle1.Offset(rowoffset:=1)     Loop  End Sub 

The main difference is it's not looping over the entire column, but instead starts at the top, and uses the offset method until there are no more rows (with data).

Of course, you'll need to change the starting cell for needle1 and needle2.

I ran this with the EFF large word list copied into both sheets, and it ran in about 4 minutes (which was less time than with @AJD, but that might've been a fluke). YMMV.

Answers 5

Just one more option. Not much different from any suggestions above ... The concept is to speed up processing by minimizing VBA - Excel interactions by loading the values to arrays and processing arrays like this:

Dim cel as String, cel2 as String Dim arr1() as String, arr2 As String  arr1 = Sheets(1).Range("A2:A" & Cells(Rows.Count, 1).End(xlUp).Row) arr2 = Sheets(2).Range("A2:A" & Cells(Rows.Count, 1).End(xlUp).Row)  For Each cel In arr1     For Each cel2 in arr2         If InStr(1, cel, cel2, 1) > 0 Then               Debug.Print cel         End If     Next cel2 Next cel 

I'd like to know if it helps at all :)

Read More