NodeJS Express pagination with Google Datastore how to integrate cursor queries with UI control

I am stuck on implementing Pagination and I just need a bit of help, either some example code or even just a hint to help me proceed in the right direction.

I'm looking for some guidance on how to integrate Google Datastore database cursors with front-end UI pagination controls. I know how to build an angular pagination service, but that's with retrieving all the data at once and due to performance issues (5,000 records+) I want to use cursors to retrieve data in subsets.

NOTE: There's a similar question here, but I need more detail than this accepted answer provides. Node pagination in google datastore

QUESTION: How can I integrate the paginated datastore cursor queries with the front-end UI controls to allow the user to select the current page and control number of results displayed on each page?

I need to build a page that displays a large number of records with dynamic pagination. The user must be able to select the number of records display on each page.

Since there are several thousand records that might be returned at one time, I want to use cursors to retrieve subsets of data.

There is an example of how to paginate in the docs, but it is a pretty basic example and does not demonstrate how to integrate with front-end UI controls.

Can anyone provide a more detailed example and/or point me in the right direction on where to begin with this requirement? Unfortunately I haven't been able to find any detailed examples online.

https://googlecloudplatform.github.io/google-cloud-node/#/docs/datastore/1.1.0/datastore

Paginating Records

var express = require('express'); var app = express();  var NUM_RESULTS_PER_PAGE = 15;  app.get('/contacts', function(req, res) {   var query = datastore.createQuery('Contacts')     .limit(NUM_RESULTS_PER_PAGE);    if (req.query.nextPageCursor) {     query.start(req.query.nextPageCursor);   }    datastore.runQuery(query, function(err, entities, info) {     if (err) {       // Error handling omitted.       return;     }      // Respond to the front end with the contacts and the cursoring token     // from the query we just ran.     var frontEndResponse = {       contacts: entities     };      // Check if  more results may exist.     if (info.moreResults !== datastore.NO_MORE_RESULTS) {       frontEndResponse.nextPageCursor = info.endCursor;     }      res.render('contacts', frontEndResponse);   }); }); 

1 Answers

Answers 1

One thing to keep in mind is the first item on the Limitations of cursors list:

• A cursor can be used only by the same project that performed the original query, and only to continue the same query. It is not possible to retrieve results using a cursor without setting up the same query from which it was originally generated.

So you need to always be able to re-create the original query inside your handler, which means you need to pass around the equivalent of your NUM_RESULTS_PER_PAGE value from one request to another. You also need to reset the query every time that value changes - meaning you can't continue browsing from where you were after changing the number of results displayed per page.

Then, to be able to use the pagination, you also need to pass around the current cursor value from one request to another, updated at every request.

Now I'm not a NodeJS user, so I can tell exactly how this passing values around from one request to another would typically be implemented. In your code req.query.nextPageCursor and frontEndResponse.nextPageCursor appear to be intended for this, but I can't tell if that's OK or not. Maybe this is a less specific question easier to find an answer for.

In python webapp2, for example, I can store such values server-side in the user's session in one request and read them from the session in a subsequent request. Donno if this is of any help.

Read More

Pagination function doesn't work in WordPress

Despite following exactly the example in the codex, the page I arrive at when clicking the pagination next link (http://localhost:3000/my_project/news/page/2/) doesn't exist ("page not found").

Why?

$paged = (get_query_var('paged')) ? get_query_var('paged') : 1; echo 'paged = ' . $paged; $regular_posts = new WP_Query('posts_per_page=3&paged=' . $paged); while ($regular_posts->have_posts()): $regular_posts->the_post();    the_title(); endwhile; echo get_next_posts_link('Older Entries', $regular_posts->max_num_pages); 

This code is contained in my "home.php" template, managing the "News" page which I created in dashboard and set as "Posts page" in "Reading Settings".

4 Answers

Answers 1

A static homepage is slightly different from an archive, as it page parameter instead of paged.

The Codex Pagination page includes this code for static homepages, which will actually work in all cases (i.e. even archive pages) because its checking for both parameters:

if ( get_query_var( 'paged' ) ) { $paged = get_query_var( 'paged' ); } elseif ( get_query_var( 'page' ) ) { $paged = get_query_var( 'page' ); } else { $paged = 1; } 

But if you only need it to work on the homepage, Changing your code for the $paged variable to the following should work too:

$paged = (get_query_var('page')) ? get_query_var('page') : 1; 

Answers 2

You can put this code in function.php file,

function pagination($pages = '', $range = 4) { $showitems = ($range * 2) + 1;  global $paged; if (empty($paged))     $paged = 1;  if ($pages == '') {     global $wp_query;     $pages = $wp_query->max_num_pages;     if (!$pages) {         $pages = 1;     } }  if (1 != $pages) {     echo "<div class=\"fl-w-left main-pagination\">";     if ($paged > 2 && $paged > $range + 1 && $showitems < $pages)         echo "<a href='" . get_pagenum_link(1) . "'>&laquo; First</a>";     if ($paged > 1 && $showitems < $pages)         echo "<a href='" . get_pagenum_link($paged - 1) . "'>&lsaquo; Previous</a>";     echo '<ul>';     for ($i = 1; $i <= $pages; $i++) {          if (1 != $pages && (!($i >= $paged + $range + 1 || $i <= $paged - $range - 1) || $pages <= $showitems )) {             echo ($paged == $i) ? "<li class=\"active\">" . $i . "</li>" : "<a href='" . get_pagenum_link($i) . "'>" . $i . "</a>";         }     }     echo '</ul>';      if ($paged < $pages && $showitems < $pages)         echo "<a href=\"" . get_pagenum_link($paged + 1) . "\">Next &rsaquo;</a>";     if ($paged < $pages - 1 && $paged + $range - 1 < $pages && $showitems < $pages)         echo "<a href='" . get_pagenum_link($pages) . "'>Last &raquo;</a>";     echo "</div>\n"; }} 

Also, put this code in your page or post template file,

<?php if (function_exists("pagination")) { ?>                     <?php pagination($post_query->max_num_pages); ?>                 <?php }                 ?> 

Answers 3

Changing $wp_query->posts_per_page in the template with a new instance of WP_Query doesn't work because the requested page number has already been validated against $wp_query->max_num_pages by WordPress before the template is executed. As the requested page is more than this value for the main query, the 404.php template is served instead, and your code in the home.php template isn't even executed.

The trick is to get $wp_query->max_num_pages to match the true maximum number of pages by manipulating the main query to use your custom arguments, before the main query is performed. $wp_query->posts_per_page and $wp_query->found_posts will then match your results, resulting in a $wp_query->max_num_pages that does too.

The best way, in my opinion, is to edit the main query arguments before the main query gets its posts. For example, you could add the following to your theme's functions.php:

function custom_main_query($query){   if(is_admin()) return;   if(!$query->is_main_query()) return;   if(is_home()) $query->set('posts_per_page', 3); }  add_action('pre_get_posts', 'custom_main_query'); 

This will then mean a standard loop will work out the box too:

// Standard loop while (have_posts()) : the_post();   the_title(); endwhile;  // Navigation links previous_posts_link('Newer Entries'); next_posts_link('Older Entries'); 

If you are only manipulating the number of posts per page, it may be better to change the default posts per page by going to Options => Reading in WordPress admin and setting Blog pages show at most to 3 here instead.

Answers 4

This error usually occurs due to limits set on the number of posts to show on your templates. It's possible this is set to less than 4. So get to your WordPress dashboard and navigate to Settings->Reading. Look for Blog pages show at most and if the value is less than 4, increase it to the total number of posts to show on your template, plus the pagination link (in your case, this should be at least 4).

Read More

Is ODATA using Microsoft Web API really REST architecture?

The more I research about Microsoft framework on ODATA I tend to believe that it is not suited for enterprise application. The framework expects all the database to be directly exposed as ViewModel, even for simple operations like Pagination & sorting.

We would be forced to use stasteful mechanism to persist page numbers rendered to the JavaScript client.

Or am I not understanding Microsoft implmentation of OData correctly?

EDIT-1:

Is ODATA V4 a Stateful Architecture? As promoted by Microsoft patterns team. I do not see any easy path of Migration from Asp.Net Web API (REST) to OData (Sounds STATEFUL) Architecture.

EDIT-2: Paging, sorting & grouping is part of incoming request from the client.

1 Answers

Answers 1

In short the MS Odata server side implementation is not statefull and it can be considered a REST architecture.

We would be forced to use stasteful mechanism to persist page numbers rendered to the JavaScript client

You provide paging information in the request. For example, if you wanted 10 items of page 2 you would take the top 10 and skip 10.

odata-url/?$count=true&$top=10&$skip=10 

As you can see the client/caller specifies the paging, there is no need for the server to track state of the client.

Also adding $count=true will return the total number of records based on the passed in filter included in the result set (in the above example there is no filter). This will allow the client to calculate the number of pages there are.

---

The framework expects all the database to be directly exposed as ViewModel...

Also not true. You can return an IQueryable<T> where T is your type. T does not have to be an EF model. For example,returning the following from a DbContext is acceptable.

public IQueryable<SomeEntity> Get() {     return dbContext.SomeEntities         .Where(x => optionalPreFiltereExpression)         .Select(x => new SomeDTO(){             Prop1 = x.Prop1,             Collection1 = x.CollectionOfInterest,             // etc         }); } 

To further illustrate that point you could also return a hard coded list of objects, although this might not be very likely in production.

public IQueryable<SomeEntity> Get() {     return new List<SomeDTO>(){         new SomeDTO(){             Prop1 = 5,             Prop2 = "Hi there"             // etc},         new SomeDTO(){             Prop1 = 6,             Prop2 = "Goodbye"             // etc}         }).AsQueryable(); } 

---

There are many resources on all the options for OData. I am not going to include everything here, otherwise I might as well just be creating a 2nd set of documentation.

• Supporting OData Query Options in ASP.NET Web API
• odata.org - getting started
• Create an OData v4 Endpoint Using ASP.NET Web API 2.2
• odata.org - OData Version 4.0. Part 1

Read More

Lists and pagination authorization in GraphQL business layer

In Dan Schafer's excellent "GraphQL at Facebook" talk from React Europe he goes over how centralizing authorization in business layer models avoids the problem of having to duplicate authorization logic for every edge that leads to an authorized node.

This works fine for something like Todo.getById(1) which in my case eventually ends up querying a database for SELECT * from todos WHERE id=1 and then verifying authorization with checkCanSee(resultFromDatabase).

However, let's say my todos table now contains 100,000 todos from multiple users, performing authorization purely in the business layer becomes impractical as I'd need to fetch every todo, filter the result using the shared authorization logic and then slicing that to perform pagination.

Am I wrong thinking that the only way to solve this is by letting authorization logic reside in the persistence layer itself?

2 Answers

Answers 1

I think one of the takeaways from Dan’s talk is the difference in how authorization is handled with GraphQL, as opposed to a typical REST endpoint.

In REST, each resource is typically associated with a single endpoint. When a request is made to that endpoint, it makes sense to check whether the requestor is authorized before processing the request. With GraphQL we may be fetching multiple resources within the same request, so this behavior is no longer desirable. As Dan puts it:

We don’t want to completely blow up the request if you can’t see one of [the requested resources].

So the preferred approach with GraphQL is to implement some kind of per-node mechanism for authorization, and to only return the resources the requester is authorized to see. And that is exactly what the example in the talk shows – one way of doing that.

If you store your to-dos in a SQL database table, it would make perfect sense for your code to just make a query like SELECT * from todos WHERE creator_id=${viewer.id} and omit using a function like checkCanSee altogether.

Similarly, you can bake pagination right into your query with limit-offset, cursors, etc. And yes, since you’re now letting your DB do the heavy lifting, you could say that we’ve moved into the persistence layer. However, it’s still up to your business logic to take the request, sanitize the inputs, construct an appropriate query and return the results in a form GraphQL can use.

I can’t speak for Dan, but I imagine his intent was not to suggest this was the only (or even optimal) way to implement authorization for a node. I think the bigger point is that if you are, for example, fetching:

{   header   todos  {     description   }   quoteOfTheDay } 

even an unauthorized client should still get a response back from the server that it can then use to render a page for the end-user (even if that response includes an empty array of to-dos).

Answers 2

You can query based on authorization results. In your Todo example:

1. Ask the authorization server whose todos you're allowed to see,
2. SELECT * FROM todos WHERE owner IN [<permitted owners]]

Read More

Multiple pagination (ajax) not working for django-el-pagination

I have 2 querysets: Post and Comment. I'm using django-el-pagination to render these using ajax.

Here's my view:

def profile(request, user, extra_context=None):      profile = Profile.objects.get(user__username=user)      page_template = 'profile.html'      if request.is_ajax():         user_queryset = request.GET.get('user_queryset')         print('Queryset:', user_queryset)         if user_queryset == 'user_posts':             page_template = 'user_posts.html'         elif user_queryset == 'user_comments':             page_template = 'user_comments.html'         else:             pass      print('Template:', page_template)      user_posts = Post.objects.filter(user=profile.user).order_by('-date')     user_comments = Comment.objects.filter(user=profile.user).order_by('-timestamp')      context = {'user_posts': user_posts,'user_comments': user_comments, 'page_template': page_template}      if extra_context is not None:         context.update(extra_context)      return render(request, page_template, context) 

I have an ajax call that find out which query set is being used. So when 'more comments' or 'more posts' (in the template) is being clicked to get more paginated objects, I know which queryset it's from. However when I use the above code and click 'more' for the ajax pagination, it appends the whole page, not the relevant child template (user_posts.html or user_comments.html). But the if request.is_ajax() code block works fine; it prints the correct template to use so this shouldn't be happening.

When I change that code block to this

if request.is_ajax():     page_template = 'user_posts.html' 

The ajax pagination for Post works. However I'd like to add ajax pagination for Comment aswell. Why doesn't my initial if request.is_ajax() work and how can I fix it?

EDIT:

Output of when I click on on more posts:

Queryset: None Template: profile.html Queryset: user_posts Template: user_posts.html 

js

$('body').on('click', '.endless_more', function() {     console.log($(this).html()); #works successfully      var user_queryset;     if ($(this).html() === 'more posts') {         console.log('POSTS'); #works successfully          var user_queryset = 'user_posts'     } else if ($(this).html() === 'more user comments') {         user_queryset = 'user_comments';         console.log('COMMENTS'); #works successfully      } else {         console.log('none');     }     $.ajax({         type: 'GET',         url: window.location.href,         data: {             'user_queryset': user_queryset         }      }) }); 

profile.html

<!--posts--> <div class="user_posts_div">     <div class="endless_page_template">         {% include "user_posts.html" %}     </div> </div>  <!--comments--> <div class="user_comments_div">     <div class="endless_page_template">         {% include "user_comments.html" %}     </div> </div> 

user_posts.html (child template)

{% paginate 5 user_posts %}     {% for post in user_posts %}         <div class="user_post">             <p class="user_post_title_p"><a class="user_post_title" href="{% url 'article' category=post.entered_category id=post.id %}">{{ post.title }}</a></p>             <p class="user_post_category">/{{ post.entered_category }}</p>             <p class="user_post_date">{{ post.date|timesince }}</p>         </div>      {% endfor %} {% show_more 'more posts' '...' %} 

2 Answers

Answers 1

What is the output of below line?

print('Queryset:', user_queryset) 

I think you have problem in below line

user_queryset = request.GET.get('user_queryset') 

this is not returning correct get parameter value for match with condition of post and comment part.

Answers 2

Could you check in your javascript near :

   user_queryset = 'user_comments'; 

Try to change it to :

   var user_queryset = 'user_comments'; 

I assume, if you go directly to comments, that the variable user_queryset will be undefined and it doesn't get passed as 'user_comments'.

Read More

Adding ajax load more button to my front page

I'm using a pre made wordpress theme for my site. However, I wanted to make a custom front-page.php so I did, but now the problem is that I can't figure out how to add the ajax load more button to it. My theme already utilizes the ajax load more button, so I thought it would be simple to add. But I think I may be adding in the code at the wrong spot, or have my queries messed up?

Can anyone help me add this load more button?

my custom front-page.php

<?php       get_header();      get_template_part ('inc/carousel');       $the_query = new WP_Query( [          'posts_per_page' => 13,          'paged' => get_query_var('paged', 1)      ] );       if ( $the_query->have_posts() ) { ?>          <div id="ajax">          <?php              $i = 0;              $j = 0;              while ( $the_query->have_posts() ) {                  $the_query->the_post();                   if ( $i % 5 === 0 ) { // Large post: on the first iteration and every 7th post after... ?>                      <div class="row">                          <article <?php post_class( 'col-sm-12 col-md-12' ); ?>>                              <div class="large-front-container">                                  <?php the_post_thumbnail('full', array('class' => 'large-front-thumbnail')); ?>                              </div>                              <div class="front-page-date"><?php echo str_replace('mins', 'minutes', human_time_diff( get_the_time('U'), current_time('timestamp') ) . ' ago'); ?></div>                             <h2><a class="front-page-post-title" href="<?php the_permalink(); ?>"><?php the_title(); ?></a></h2>                              <p class="front-page-post-excerpt"><?php echo get_the_excerpt(); ?></p>                              <div class="front-page-post-info">                                  <a class="moretext" href="<?php the_permalink(); ?>">Read more</a>                                  <?php get_template_part( 'front-shop-the-post' ); ?>                                  <?php get_template_part( 'share-buttons' ); ?>                                  <div class="front-comments"><?php comments_popup_link ('0', '1', '%', 'comment-count', 'none'); ?></div>                              </div>                          </article>                      </div>                  <?php } else { // Small posts ?>                      <?php if($j % 2 === 0) echo '<div class="row">'; ?>                          <article <?php post_class( 'col-sm-6 col-md-6' ); ?>>                              <?php the_post_thumbnail('full', array('class' => 'medium-front-thumbnail')); ?>                              <div class="front-page-date"><?php echo human_time_diff( get_the_time('U'), current_time('timestamp') ) . ' ago'; ?></div>                             <h2><a class="front-page-post-title" href="<?php the_permalink(); ?>"><?php the_title(); ?></a></h2>                              <p class="front-page-post-excerpt"><?php echo get_the_excerpt(); ?></p>                              <div class="front-page-post-info">                                  <a class="moretext" href="<?php the_permalink(); ?>">Read more</a>                                 <?php get_template_part( 'front-shop-the-post' ); ?>                                 <?php get_template_part( 'share-buttons' ); ?>                                 <div class="front-comments"><?php comments_popup_link ('0', '1', '%', 'comment-count', 'none'); ?></div>                              </div>                         </article>                  <?php $j++; if($j % 2 === 0) echo '</div>'; ?>          <?php          }          $i++;          }?>          </div>      <?php     }      get_footer(); 

the post-nav.php code that I found inside my theme

<div class="row pagination-below"><div class="col-md-12">     <?php      $pagination_type = novablog_getVariable('pagination_type') ? novablog_getVariable('pagination_type') : 'pagnum';     if($pagination_type=='pagnum') :          the_posts_pagination( array(             'mid_size' => 3,             'type' => 'list',             'prev_text'          => theme_locals("prev"),             'next_text'          => theme_locals("next")         ) );     endif;      global $wp_query;     if ( $wp_query->max_num_pages > 1 && $pagination_type=='paglink' ) : ?>         <div class="paglink">             <span class="pull-left"><?php previous_posts_link(theme_locals("newer")) ?></span>                    <span class="pull-right"><?php next_posts_link(theme_locals("older")) ?></span>         </div>     <?php endif; ?>      <?php         if ( $wp_query->max_num_pages > 1 && $pagination_type=='loadmore' or $wp_query->max_num_pages > 1 && $pagination_type=='infinite' ) {              $all_num_pages = $wp_query -> max_num_pages;             $next_page_url = novablog_next_page($all_num_pages);     ?>             <div class="ajax-pagination-container">               <a href="<?php echo esc_url($next_page_url); ?>" id="ajax-load-more-posts-button"></a>             </div>     <?php } ?> </div></div> 

This is how the load more button appears on my local host

example of what I want my front page post layout to look like. 1 post on a row, 2 rows of 2 posts on a row, 1 post on a row, and so on. Then after every 15 posts the load more button appears.

This is what chrome developer looks like when I inspect the load more button

2 Answers

Answers 1

You should probably just use an appropriate plugin to provide the functionality you're stuggling with.

This https://en-ca.wordpress.org/plugins/easy-load-more/ claims to do exactly what you're looking for with "minimal" theme changes.

Answers 2

Add this to the front-page.php

<?php  get_header(); get_template_part ('inc/carousel');  ?>   <script>     var now=2; // when click start in page 2      jQuery(document).on('click', '#load_more_btn', function () {          jQuery.ajax({             type: "POST",             url: "<?php echo get_site_url(); ?>/wp-admin/admin-ajax.php",             data: {                 action: 'my_load_more_function', // the name of the function in functions.php                 paged: now, // set the page to get the ajax request                 posts_per_page: 1  //number of post to get (use 1 for testing)             },             success: function (data) {                 jQuery("#ajax").append(data);  // put the content into ajax container                 now=now+1; // add 1 to next page             },             error: function (errorThrown) {                 alert(errorThrown); // only for debuggin             }         });     }); </script>  <section id="ajax"><!-- i have to change div to section, maybe a extra div declare --> <?php  $the_query = new WP_Query( [     'posts_per_page' => 1, // i use 1 for testing     'orderby'=>'title', // add order for prevent duplicity     'order'=>'ASC',     'paged' => get_query_var('paged', 1) //page number 1 on load ] );  if ($the_query->have_posts()) {          $i = 0;         $j = 0;         while ($the_query->have_posts()) {             $the_query->the_post();              if ( $i % 5 === 0 ) { // Large post: on the first iteration and every 7th post after... ?>                 <div class="row">                     <article <?php post_class( 'col-sm-12 col-md-12' ); ?>>                         <div class="large-front-container">                             <?php the_post_thumbnail('full', array('class' => 'large-front-thumbnail')); ?>                         </div>                         <div class="front-page-date"><?php echo str_replace('mins', 'minutes', human_time_diff( get_the_time('U'), current_time('timestamp') ) . ' ago'); ?></div>                         <h2><a class="front-page-post-title" href="<?php the_permalink(); ?>"><?php the_title(); ?></a></h2>                         <p class="front-page-post-excerpt"><?php echo get_the_excerpt(); ?></p>                         <div class="front-page-post-info">                             <a class="moretext" href="<?php the_permalink(); ?>">Read more</a>                             <?php get_template_part( 'front-shop-the-post' ); ?>                             <?php get_template_part( 'share-buttons' ); ?>                             <div class="front-comments"><?php comments_popup_link ('0', '1', '%', 'comment-count', 'none'); ?></div>                         </div>                     </article>                 </div>             <?php } else { // Small posts ?>                 <?php if($j % 2 === 0){ echo '<div class="row">';} ?>                 <article <?php post_class( 'col-sm-6 col-md-6' ); ?>>                     <?php the_post_thumbnail('full', array('class' => 'medium-front-thumbnail')); ?>                     <div class="front-page-date"><?php echo human_time_diff( get_the_time('U'), current_time('timestamp') ) . ' ago'; ?></div>                     <h2><a class="front-page-post-title" href="<?php the_permalink(); ?>"><?php the_title(); ?></a></h2>                     <p class="front-page-post-excerpt"><?php echo get_the_excerpt(); ?></p>                     <div class="front-page-post-info">                         <a class="moretext" href="<?php the_permalink(); ?>">Read more</a>                         <?php get_template_part( 'front-shop-the-post' ); ?>                         <?php get_template_part( 'share-buttons' ); ?>                         <div class="front-comments"><?php comments_popup_link ('0', '1', '%', 'comment-count', 'none'); ?></div>                     </div>                 </article>                 <?php $j++; if($j % 2 === 0){ echo '</div>';}?>                 <?php             }             $i++;         }?>     <?php }?> </section>  <button id="load_more_btn">Load More</button> <!-- button out of ajax container for load content and button displayed at the bottom --> <?php get_footer(); 

And then in the functions.php add this code below:

add_action('wp_ajax_my_load_more_function', 'my_load_more_function'); add_action('wp_ajax_nopriv_my_load_more_function', 'my_load_more_function');  function my_load_more_function() {      $query = new WP_Query( [         'posts_per_page' => $_POST["posts_per_page"],         'orderby'=>'title',         'order'=>'ASC',         'paged' => get_query_var('paged', $_POST["paged"])     ] );       if ($query->have_posts()) {          $i = 0;         $j = 0;          while ($query->have_posts()) {                 $query->the_post();              if ( $i % 5 === 0 ) { // Large post: on the first iteration and every 7th post after... ?>                 <div class="row">                     <article <?php post_class( 'col-sm-12 col-md-12' ); ?>>                         <div class="large-front-container">                             <?php the_post_thumbnail('full', array('class' => 'large-front-thumbnail')); ?>                         </div>                         <div class="front-page-date"><?php echo str_replace('mins', 'minutes', human_time_diff( get_the_time('U'), current_time('timestamp') ) . ' ago'); ?></div>                         <h2><a class="front-page-post-title" href="<?php the_permalink(); ?>"><?php the_title(); ?></a></h2>                         <p class="front-page-post-excerpt"><?php echo get_the_excerpt(); ?></p>                         <div class="front-page-post-info">                             <a class="moretext" href="<?php the_permalink(); ?>">Read more</a>                             <?php get_template_part( 'front-shop-the-post' ); ?>                             <?php get_template_part( 'share-buttons' ); ?>                             <div class="front-comments"><?php comments_popup_link ('0', '1', '%', 'comment-count', 'none'); ?></div>                         </div>                     </article>                 </div>             <?php } else { // Small posts ?>                 <?php if($j % 2 === 0) echo '<div class="row">'; ?>                 <article <?php post_class( 'col-sm-6 col-md-6' ); ?>>                     <?php the_post_thumbnail('full', array('class' => 'medium-front-thumbnail')); ?>                     <div class="front-page-date"><?php echo human_time_diff( get_the_time('U'), current_time('timestamp') ) . ' ago'; ?></div>                     <h2><a class="front-page-post-title" href="<?php the_permalink(); ?>"><?php the_title(); ?></a></h2>                     <p class="front-page-post-excerpt"><?php echo get_the_excerpt(); ?></p>                     <div class="front-page-post-info">                         <a class="moretext" href="<?php the_permalink(); ?>">Read more</a>                         <?php get_template_part( 'front-shop-the-post' ); ?>                         <?php get_template_part( 'share-buttons' ); ?>                         <div class="front-comments"><?php comments_popup_link ('0', '1', '%', 'comment-count', 'none'); ?></div>                     </div>                 </article>                 <?php $j++; if($j % 2 === 0) echo '</div>'; ?>                 <?php             }             $i++;          }         wp_reset_query();     }      exit; } 

I think your loop settings in front-page.php have issues to resolve, in the posts-per-page parameter, displays the posts-per-page + 1 results.

Let me know if this resolve your question.

Read More

One pagination for multiple queries (or how to group a one query)

I have multiple queries in the same page, like this:

$args_1 = array (     'category_name' => get_query_var( 'category_name' ),     'post__in'      => $sticky = get_option( 'sticky_posts' );     'orderby' => 'date',     'order' => 'DESC'     );  $sticky_query = new WP_Query ($args_1);  // loop   $args_2 = array(     'category_name' => get_query_var( 'category_name' ),     'post__not_in'  => get_option( 'sticky_posts' ),     'category__not_in' => array(11114),     'orderby' => 'date',     'order' => 'DESC' );  $query_2 = new WP_Query ($args_2);  // loop    $args_3 = array(     'category_name' => get_query_var( 'category_name' ),     'post__not_in'  => get_option( 'sticky_posts' ),     'category__in' => array(11114),     'orderby' => 'date',     'order' => 'DESC' );   $query_3 = new WP_Query($args_3 );  // loop 

I would:

1) limit the total of posts per page to 15

2) (I can't do this, I've searched anywhere but I didn't find a solution) Make this "combined" pagination works . Now, and it's logical, the second page is the same as the first page...

Or the solution can be to make only one query and group the posts and order the groups instead?

3 Answers

Answers 1

Final Solution

1. you don't need custom pagination if you merge two queries and you don't use "offset", so I've removed

   $big = 999999999; // need an unlikely integer  echo paginate_links( array(     'base' => str_replace( $big, '%#%', esc_url( get_pagenum_link( $big  ) ) ),     'format' => '?paged=%#%',     'current' => max( 1, get_query_var('paged') ),     'total' => $the_query->max_num_pages ) ); 

And added:

    echo paginate_links();  

2. You have to set 'order' to 'post__in' value to keep the order given to the posts in the separate queries.

3. To avoid memory errors (like '"Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 262144 bytes)".') caused by queries that hold many posts, you have to retrive only post ids in the queries. See here.

Here's the final code:

    global $wp_query;        $paged = ( get_query_var( 'paged' ) ) ? get_query_var( 'paged' ) : 1;     $post_per_page = 35; // How many post per page - setup as you need     $nr_of_posts_to_be_analyzed_to_include_only_current_events = 100; // How many post to analyze to include only current events, ie events that are not in category id 11114        $sticky = get_option( 'sticky_posts' );      // posts in category, limited in number otherwise - if you set '-1' and you have 7.000 posts - you exceed the script time limit execution      $category_posts = get_posts(array('category_name' => get_query_var( 'category_name' ), 'posts_per_page' => $nr_of_posts_to_be_analyzed_to_include_only_current_events,'orderby' => 'date', 'order' => 'DESC'));      $category_posts_ids = array();      foreach( $category_posts as $post ) {          $category_posts_ids[]=$post->ID; // Array with posts ID      }     // var_dump($category_posts_ids);      // expired posts in category, limited in number otherwise - if you set '-1' and you have 7.000 posts - you exceed the script time limit execution     $expired_posts = get_posts(array('category__and' => array(11114, get_query_var('cat')), 'posts_per_page' => $nr_of_posts_to_be_analyzed_to_include_only_current_events, 'orderby' => 'date', 'order' => 'DESC'));     $expired_posts_ids = array();     foreach( $expired_posts as $post ) {          $expired_posts_ids[]=$post->ID; // Array with posts ID     }       // first 100 posts in this category per page - the expired posts that are in 100 first posts in this category gives as result an array of not expired posts    $array_where_to_get_post =  array_diff( $category_posts_ids, $sticky, $expired_posts_ids );       // 1/3 only the sticky posts    $stickies_posts_args = array(     'category_name' => get_query_var( 'category_name' ),     'post__in'  => get_option( 'sticky_posts' ),     'orderby' => 'date',     'order' => 'DESC',     'posts_per_page' => -1,     'fields'         => 'ids' // important: to avoid memory exhausted error we retrieve postids only     );     // 2/3 only the not expired and not sticky posts    $not_expired_posts_args = array(     'post__in' => $array_where_to_get_post,     'orderby' => 'date',     'order' => 'DESC',     'posts_per_page' => -1,     'fields'         => 'ids', // important: to avoid memory exhausted error we retrieve postids only,     'suppress_filters' => false     );     // 3/3 only the expired and not sticky posts    $expired_posts_args = array(     'category_name' => get_query_var( 'category_name' ),     'post__not_in'  => get_option( 'sticky_posts' ), // not sticky posts     'category__in' => array(11114), // expired posts     'orderby' => 'date',     'order' => 'DESC',     'posts_per_page' => -1,     'fields'         => 'ids' // important: to avoid memory exhausted error we retrieve postids only    );     $firstQuery = get_posts($stickies_posts_args);    add_filter('posts_join','join_posts_and_events'); // apply some filter   add_filter('posts_orderby', 'order_posts'); // apply some filter   $secondQuery = get_posts($not_expired_posts_args);   remove_filter('posts_join','join_posts_and_events');   remove_filter('posts_orderby', 'order_posts');    $thirdQuery = get_posts($expired_posts_args);     $mergePosts = array_merge( $firstQuery, $secondQuery, $thirdQuery ); // Merge all queries    $uniquePosts = array_unique($mergePosts); // Create an array with unique posts     // Final query    $args = array(     'post_type' => 'any',        'post__in' => $uniquePosts,     'paged' => $paged,     'orderby' => 'post__in', // order the posts as they are (already ordered)     'order' => 'DESC',     'posts_per_page' => $post_per_page,     'ignore_sticky_posts' => 1 // otherwise it breaks pagination    );    $wp_query = new WP_Query($args);      if ( $wp_query->have_posts() ) :         while ( $wp_query->have_posts() ) { $wp_query->the_post();     // outputs what you want    endwhile;     endif;    echo paginate_links();  

Answers 2

Add below parametter into each query for paginating

'paged' => get_query_var('<'page' or something else you defined>') 

EDIT: Hmm, maybe this work!

$query1 = new WP_Query($args1); // Has above paginate param $query1Ids = get_all_id_in_query1($query1);  $args2 = array(     'post__not_in' => $query1Ids,     // Other param ); $query2 = new WP_Query($args2); // Have above paginate param $query2Ids = get_all_id_in_query2($query2);  $args3 = array(     'post__not_in' => array_merge($query1Ids, $query2Ids),     // Other param, also have paginate param ); $query3 = new WP_Query($args3); 

Loop each query and show your posts.

Answers 3

Try this one below, I merge two queries (post and page). Working for me - display all posts/page with pagination.

<?php   $paged = get_query_var('paged') ? get_query_var('paged') : 1; $post_per_page = 5; // How many post per page - setup as you need  $firstQuery = new WP_Query('post_type=post'); // First query - you should change args $secondQuery = new WP_Query('post_type=page'); // Second query - you should change args   $post_ids = array_merge( $firstQuery, $secondQuery ); // Merge all queries  $query = new WP_Query(     array(         'post_type'      => array('post', 'page'), // You should change post types as you need         'post__in'       => $post_ids,          'paged'          => $paged,         'orderby'        => 'date',          'order'          => 'DESC',         'posts_per_page' => $post_per_page     ) );  $totalPosts = $firstQuery->post_count + $secondQuery->post_count; // Count posts   if( $query->have_posts() ):      while( $query->have_posts() ): $query->the_post();          echo the_title()."</br>"; // Content      endwhile;       wp_reset_query();  endif;   $big = 999999999; // need an unlikely integer  echo paginate_links( array(     'base' => str_replace( $big, '%#%', esc_url( get_pagenum_link( $big ) ) ),     'format' => '?paged=%#%',     'current' => max( 1, get_query_var('paged') ),     'total' => $totalPosts / $post_per_page, // Total pages ) );  ?> 

New Solution - Edit

Just adjust the first and second query

<?php   global $wp_query;  $paged = ( get_query_var( 'paged' ) ) ? get_query_var( 'paged' ) : 1; $post_per_page = 5; // How many post per page - setup as you need  // First query $firstQuery = get_posts(array(     'posts_per_page' => -1,     'category_name' => get_query_var( 'category_name' ),     'post__in'      => $sticky = get_option( 'sticky_posts' )  ));  // Second query  $secondQuery = get_posts(array(     'posts_per_page' => -1,     'category_name' => get_query_var( 'category_name' ),     'post__not_in'  => get_option( 'sticky_posts' ), ));   $mergePosts = array_merge( $firstQuery, $secondQuery ); // Merge all queries  $postIds = array();  foreach( $mergePosts as $post ) {      $postIds[]=$post->ID; // Array with posts ID  }  $uniquePosts = array_unique($postIds); // Create an array with unigue posts   // Final query $args = array( 'post_type' => 'any',    'post__in' => $uniquePosts, 'paged' => $paged, 'orderby' => 'date', 'order' => 'DESC', 'posts_per_page' => $post_per_page, );   $the_query = new WP_Query($args);  if( $the_query->have_posts() ):      while( $the_query->have_posts() ): $the_query->the_post();          echo the_title()."</br>"; // Content      endwhile;       wp_reset_query();  endif;  $big = 999999999; // need an unlikely integer  echo paginate_links( array(     'base' => str_replace( $big, '%#%', esc_url( get_pagenum_link( $big ) ) ),     'format' => '?paged=%#%',     'current' => max( 1, get_query_var('paged') ),     'total' => $the_query->max_num_pages ) );  ?>   

Read More

Paging in ASP.Net; the number of pages never changes after filtering

The issue comes when you open a page with only 1 record. It fills the NavMenu with 3 links; "First", "1" and "Last". For some reason, when you run a search query that will return more than one page, it still only displays "First", "1" and "Last". Similarly, if you start with 4 pages and your subsequent search query only returns 2 records, it still shows "First", "1", "2", "3", "4" and "Last". So, for some reason, however many pages you start with, you'll always get. How can you reset the page counter/display?

Here's my C# code-behind:

public void RunTheSearch() {     //Run the Stored Procedure first     SqlConnection connection2 = new SqlConnection(strCon1);     SqlCommand cmd2 = new SqlCommand();     cmd2.CommandType = CommandType.StoredProcedure;     cmd2.CommandText = "sp_Search";     cmd2.Connection = connection2;      //--- A bunch of code that returns a dataset.  Lengthy and unnecessary to my issue      connection2.Open();      SqlDataAdapter adp = new SqlDataAdapter(cmd2);       DataSet ds = new DataSet();     adp.Fill(ds, "OLDPages");      //Pagination code so only a set number of records loads at a time.     //  Done to speed up the loading, since this list gets really long.     PagedDataSource pds = new PagedDataSource();     pds.DataSource = ds.Tables["OLDPages"].DefaultView;      pds.AllowPaging = true;     pds.PageSize = 10;     //NavMenu.Items.Clear();      int currentPage;      if (Request.QueryString["page"] != null)     {     currentPage = Int32.Parse(Request.QueryString["page"]);     }     else     {     currentPage = 1;     }      pds.CurrentPageIndex = currentPage - 1;     //Label1.Text = "Page " + currentPage + " of " + pds.PageCount;       if (!pds.IsFirstPage)     {     MenuItem itemMessage = NavMenu.FindItem("First");     itemMessage.NavigateUrl = Request.CurrentExecutionFilePath + "?page=1";     }      AcctRepeater.DataSource = pds;     AcctRepeater.DataBind();      CreatePagingControl(pds.PageCount, pds.CurrentPageIndex);     // End of Pagination code      connection2.Close(); }  private void CreatePagingControl(int PCount, int PIndex) {     int PIndex2 = 0;     int SCounter = PIndex + 1;     int RowCount = PCount;      //Allow the pagination menu to always start 5 less than the current page you're on     if (PIndex < 5)     {     PIndex2 = 0;     }     else     {     PIndex2 = PIndex - 5;     }      // Show 10 total page numbers.  You can increase or shrink that range by changing the 10 to whatever number you want     for (int i = PIndex2; i < PIndex2 + 10 && i < PCount; i++)     {     NavMenu.Items.Add(new MenuItem     {         Text = (i + 1).ToString(),         NavigateUrl = Request.CurrentExecutionFilePath + "?page=" + (i + 1).ToString()     });      // Now determine the selected item so the proper CSS can be applied     foreach (MenuItem item in NavMenu.Items)     {         item.Selected = item.Text.Equals(SCounter.ToString());     }     }      NavMenu.Items.Add(new MenuItem     {     Text = "Last",     NavigateUrl = Request.CurrentExecutionFilePath + "?page=" + (PCount)     }); }          

And on the aspx page:

<asp:Menu ID="NavMenu" runat="server" CssClass="menu"  IncludeStyleBlock="false" Orientation="Horizontal" width="703px" BackColor="#CC3300" EnableViewState="true"> <Items>      <asp:MenuItem NavigateUrl="~/Default.aspx" Text="First" Selectable="true" /> </Items> </asp:Menu>  

I did try NavMenu.Items.Clear(), but it didn't like that because it also cleared out the hard-coded item on the aspx side.

4 Answers

Answers 1

I cannot reproduce it.

My intuition tells me you're not posting back and that's why you need to clear() the results.

This C# code works fine.

protected void Page_Load(object sender, EventArgs e) {     RunTheSearch(); } 

Answers 2

The only problem here is with ViewState. Items are persisted in ViewState, so if you press button causing PostBack twice the items are appended at the end of items that were appended previously.

If you change the <asp:Menu ... EnableViewState="false" /> you don't need clean the items anymore.

Alternatively (if you need ViewState enabled for some other reason) you can mark the items when you're adding them like:

NavMenu.Items.Add(new MenuItem {     //...     Value = "Paging" }); 

And instead of cleaning up all items clear just the marked ones:

 var removableItems = NavMenu.Items.Cast<MenuItem>()    .Where(i => i.Value == "Paging").ToList();  foreach (var removableItem in removableItems)  {    NavMenu.Items.Remove(removableItem);  }     

Answers 3

You are retrieving the data from a stored procedure named sp_Search but your query in all runs will be same because you didn't specify any parameters in your stored procedure (based on the code that you posted). I tested your code by modifying the stored procedure and send a parameter to it and also using NavMenu.Items.Clear() as you said and it works fine for me:

Your SP should be something like this:

CREATE PROCEDURE [dbo].[sp_Search]     @param1 NVARCHAR(50) AS     SELECT * from yourTableName where SearchField = @param1 RETURN 0 

And in c#:

public void RunTheSearch(string id) {     ...     cmd2.CommandType = CommandType.StoredProcedure;     cmd2.CommandText = "sp_Search";     cmd2.Parameters.Add("@param1", SqlDbType.NVarChar, 50).Value = id;     ...     ... 

So in your Page_Load call the RunTheSearch method by passing a parameter which returns one record:

protected void Page_Load(object sender, EventArgs e) {     RunTheSearch("p1");             } 

And somewhere else call the RunTheSearch method by passing a parameter which returns multiple record and result would be more than one page:

protected void Button1_OnClick(object sender, EventArgs e) {     NavMenu.Items.Clear();     RunTheSearch("p2"); } 

Answers 4

This was eventually solved by putting the menu in an Update panel. So, on the aspx side I now have:

<div class="clear hideSkiplink" id="NavDiv" style="margin:0 auto; display: table;">     <asp:UpdatePanel ID="NavUpdatePanel" runat="server" UpdateMode="Conditional">         <ContentTemplate>             <asp:Menu ID="NavMenu" runat="server" CssClass="menu"                  IncludeStyleBlock="false" Orientation="Horizontal" width="703px"                 BackColor="#CC3300" EnableViewState="false">                 <Items>                      <asp:MenuItem NavigateUrl="~/Default.aspx" Text="First" Selectable="true" />                 </Items>             </asp:Menu>         </ContentTemplate>     </asp:UpdatePanel> </div> 

Thank you everyone who gave it a go. It was actually while replying to the posts that a switch went on in my brain and I got the idea to try it.

Read More

How to add pagination in Restangular and Django Rest Framework?

In DRF I have added pagination limit to 100 'PAGINATE_BY': 100, since Restangular expects results in array form, I had to use the below meta extractor function in my angular app module

var app = angular.module("myapp", ["restangular"].config(function(             RestangularProvider){    RestangularProvider.setResponseExtractor(function(response, operation, what, url) {     if (operation === "getList") {         var newResponse = response.results;         newResponse._resultmeta = {             "count": response.count,             "next": response.next,             "previous": response.previous         };         return newResponse;     }      return response;     }); }); 

and my controller looks like

app.controller('DataCtrl',function($scope, Restangular){      var resource = Restangular.all('myapp/api/dataendpoint/');         resource.getList().then(function(data){         $scope.records = data;     });     } 

Meta info is not available in controller, how do I paginate if there are more than 100 records available?

1 Answers

Answers 1

I suppose you could simply call:

RestangularProvider.addResponseExtractor(function(data, operation, what, url, response) {   if (operation === "getList") {       data._resultmeta = {           "count": response.count,           "next": response.next,           "previous": response.previous       };       return data;   }    return response; }); 

and

var page = 2; var resource = Restangular.all('myapp/api/dataendpoint/'); resource.getList({page: page}).then(function(data){   console.log(data._resultmeta.next ? 'there is more pages' : 'You reach the end'); }); 

I'm not usual with Rectangular but Django Rest Framework support pagination from query parameter

Read More

Is it a bad idea to store row count and number of row to speed up pagination?

My website has more than 20.000.000 entries, entries have categories (FK) and tags (M2M). As for query even like SELECT id FROM table ORDER BY id LIMIT 1000000, 10 MySQL needs to scan 1000010 rows, but that is really unacceptably slow (and pks, indexes, joins etc etc don't help much here, still 1000010 rows). So I am trying to speed up pagination by storing row count and row number with triggers like this:

DELIMITER // CREATE TRIGGER @trigger_name AFTER INSERT ON entry_table FOR EACH ROW BEGIN     UPDATE category_table SET row_count = (@rc := row_count + 1)     WHERE id = NEW.category_id;     NEW.row_number_in_category = @rc; END // 

And then I can simply:

SELECT *  FROM entry_table  WHERE row_number_in_category > 10  ORDER BY row_number_in_category  LIMIT 10 

(now only 10 rows scanned and therefore selects are blazing fast, although inserts are slower, but they are rare comparing to selects, so it is ok)

Is it a bad approach and are there any good alternatives?

1 Answers

Answers 1

Although I like the solution in the question. It may present some issues if data in the entry_table is changed - perhaps deleted or assigned to different categories over time.

It also limits the ways in which the data can be sorted, the method assumes that data is only sorted by the insert order. Covering multiple sort methods requires additional triggers and summary data.

One alternate way of paginating is to pass in offset of the field you are sorting/paginating by instead of an offset to the limit parameter.

Instead of this:

SELECT id FROM table ORDER BY id LIMIT 1000000, 10 

Do this - assuming in this scenario that the last result viewed had an id of 1000000.

SELECT id FROM table WHERE id > 1000000 ORDER BY id LIMIT 0, 10 

By tracking the offset of the pagination, this can be passed to subsequent queries for data and avoids the database sorting rows that are not ever going to be part of the end result.

If you really only wanted 10 rows out of 20million, you could go further and guess that the next 10 matching rows will occur in the next 1000 overall results. Perhaps with some logic to repeat the query with a larger allowance if this is not the case.

SELECT id FROM table WHERE id BETWEEN 1000000 AND 1001000 ORDER BY id LIMIT 0, 10 

This should be significantly faster because the sort will probably be able to limit the result in a single pass.

Read More