Showing posts with label multi-tenant. Show all posts
Showing posts with label multi-tenant. Show all posts

Thursday, June 29, 2017

Firebase multi-tenancy with play framework depends on header value of HTTP request

Leave a Comment

I have a play framework project that provides APIs that shared between multiple front-ends, currently, I'm working on single front-end but I want to create a multi-tenant backend, each front-end got its own Firebase account.

My problem that I have to consider which firebase project to access depends on the request header value, that came with different values depends on the front end.

What I have now: FirebaseAppProvider.java:

public class FirebaseAppProvider implements Provider<FirebaseApp> {       private final Logger.ALogger logger;     private final Environment environment;     private final Configuration configuration;      @Inject     public FirebaseAppProvider(Environment environment, Configuration configuration) {         this.logger = Logger.of(this.getClass());         this.environment = environment;         this.configuration = configuration;     }      @Singleton     @Override     public FirebaseApp get() {         HashMap<String, String> firebaseProjects = (HashMap<String, String>) configuration.getObject("firebase");         firebaseProjects.forEach((websiteId, projectId) -> {             FileInputStream serviceAccount = null;             try {                 serviceAccount = new FileInputStream(environment.classLoader().getResource(String.format("firebase/%s.json", projectId)).getPath());             } catch (FileNotFoundException e) {                 e.printStackTrace();                 return;             }              FirebaseOptions options = new FirebaseOptions.Builder().setCredential(FirebaseCredentials.fromCertificate(serviceAccount))                     .setDatabaseUrl(String.format("https://%s.firebaseio.com/", projectId))                     .build();               FirebaseApp firebaseApp = FirebaseApp.initializeApp(options, projectId);              logger.info("FirebaseApp initialized");         });          return FirebaseApp.getInstance();     } } 

Also for Database: FirebaseDatabaseProvider.java

public class FirebaseDatabaseProvider implements Provider<FirebaseDatabase> {      private final FirebaseApp firebaseApp;     public static List<TaxItem> TAXES = new ArrayList<>();      @Inject     public FirebaseDatabaseProvider(FirebaseApp firebaseApp) {         this.firebaseApp = firebaseApp;         fetchTaxes();     }      @Singleton     @Override     public FirebaseDatabase get() {         return FirebaseDatabase.getInstance(firebaseApp);     }      @Singleton     public DatabaseReference getUserDataReference() {         return this.get().getReference("/usersData");     }      @Singleton     public DatabaseReference getTaxesConfigurationReference() {         return this.get().getReference("/appData/taxConfiguration");     }     private void fetchTaxes() {         DatabaseReference bundlesRef = getTaxesConfigurationReference().child("taxes");         bundlesRef.addValueEventListener(new ValueEventListener() {             @Override             public void onDataChange(DataSnapshot dataSnapshot) {                 TAXES.clear();                 dataSnapshot.getChildren().forEach(tax -> TAXES.add(tax.getValue(TaxItem.class)));                 Logger.info(String.format("==> %d taxes records loaded", TAXES.size()));             }              @Override             public void onCancelled(DatabaseError databaseError) {                 Logger.warn("The read failed: " + databaseError.getCode());             }         });     } } 

So I bind them as well from Module.java:

public class Module extends AbstractModule {      @Override     public void configure() {        bind(FirebaseApp.class).toProvider(FirebaseAppProvider.class).asEagerSingleton();         bind(FirebaseAuth.class).toProvider(FirebaseAuthProvider.class).asEagerSingleton();         bind(FirebaseDatabase.class).toProvider(FirebaseDatabaseProvider.class).asEagerSingleton();     }  } 

my ActionCreator:

public class ActionCreator implements play.http.ActionCreator {      @Inject     public ActionCreator() {     }      @Override     public Action createAction(Http.Request request, Method actionMethod) {         switchTenancyId(request);         return new Action.Simple() {             @Override             public CompletionStage<Result> call(Http.Context ctx) {                 return delegate.call(ctx);             }         };     }      private void switchTenancyId(Http.RequestHeader request) {         // DO something here     }      private Optional<String> getTenancyId(Http.RequestHeader request) {         String websiteId = request.getHeader("Website-ID");         System.out.println(websiteId);         return null;     } } 

What I want is when I use Database service, or auth service, I read the website id and decide which firebase project to access, I really tried the solution like this answer here: Multi tenancy with Guice Custom Scopes and Jersey

Please note I'm willing to use differents projects, not the same firebase project for each front-end.

But kinda lost, especially the request can be only accessed from controller or ActionCreator, so what I got from the question above is load providers by key into ThreadLocal and switch them for each request depends on the annotation, but I was unable to do this because of the lack of knowledge.


The minimized version of my project can be found here: https://github.com/almothafar/play-with-multi-tenant-firebase

Also, I uploaded taxes-data-export.json file to import inside firebase project for a test.

2 Answers

Answers 1

I believe Custom Scopes for this is overkill. I would recommend doing the Request-Scoped seeding from Guice's own wiki. In your case that would be something like

public class TenancyFilter implements Filter {     @Override     public void doFilter(ServletRequest request,  ServletResponse response, FilterChain chain) throws IOException, ServletException {         HttpServletRequest httpRequest = (HttpServletRequest) request;         String tenancyId = httpRequest.getHeader("YOUR-TENANCY-ID-HEADER-NAME");         httpRequest.setAttribute(                 Key.get(String.class, Names.named("tenancyId")).toString(),                 userId         );         chain.doFilter(request, response);     }      @Override     public void init(FilterConfig filterConfig) throws ServletException { }      @Override     public void destroy() { } }; 

It has to be bound in a ServletModule

public class YourModule extends ServletModule {     @Override     protected void configureServlets() {         filter("/*").through(TenancyFilter.class);     }      @Provides     @RequestScoped     @Named("tenancyId")     String provideTenancyId() {         throw new IllegalStateException("user id must be manually seeded");     } } 

Then anywhere you need to get the Tenancy ID you just inject

public class SomeClass {     private final Provider<String> tenancyIdProvider;      @Inject     SomeClass(@Named("tenancyId") Provider<String> tenancyIdProvider) {         this.tenancyIdProvider = tenancyIdProvider;     }      // Methods in request call tenancyIdProvider.get() to get and take action based on Tenancy ID. } 

Answers 2

Right, so I know Play a lot better than FireBase, but it seems to me you want to extract a tenancy ID from the request prior to feeding this into your FrieBase backend? Context when writing Java in play is Thread local, but even when doing things async you can make sure the Http.context info goes along for the ride by injecting the execution context. I would not do this via the action creator, unless you want to intercept which action is called. (Though I have a hackish solution for that as well.)

So, after a comment I'll try to elucidate here, your incoming request will be routed to a controller, like below (let me know if you need clearing up on routing etc):

Below is a solution for caching a retrieved FireBaseApp based on a "Website-ID" retrieved from the request, though I would likely put the tenancyId in the session.

import javax.inject.Inject; import java.util.concurrent.CompletionStage;  public class MyController extends Controller {     private HttpExecutionContext ec; //This is the execution-context.     private FirebaseAppProvider appProvider;     private CacheApi cache;     @Inject     public MyController(HttpExecutionContext ec, FireBaseAppProvider provider,CacheApi cache) {         this.ec = ec;         this.appProvider = provider;         this.cache = cache;      }     /**     *Retrieves a website-id from request and attempts to retrieve      *FireBaseApp object from Cache.     *If not found a new FireBaseApp will be constructed by      *FireBaseAppProvider and cached.     **/     private FireBaseApp getFireBaseApp(){      String tenancyId = request.getHeader("Website-ID);      FireBaseApp app = (FireBaseApp)cache.get(tenancyId);      if(app==null){        app=appProvider.get();        cache.put(tenancyId,app);        }      return app;     }         public CompletionStage<Result> index() {         return CompletableFuture.supplyAsync(() -> {            FireBaseApp app = getFireBaseApp();            //Do things with app.         }, ec.current()); //Used here.     } } 

Now in FireBaseAppProvider you can access the header via play.mvc.Controller, the only thing you need is to provide the HttpExecutionContext via ec.current. So (once again, I'm avoiding anything FireBase specific), in FireBaseProvider:

import play.mvc.Controller; public class FireBaseAppProvider {   public  String getWebsiteKey(){         String website = Controller.request().getHeader("Website-ID");         //If you need to handle a missing header etc, do it here.         return website;     }   public FireBaseApp get(){      String tenancyId = getWebsiteKey();      //Code to do actual construction here.   } } 

Let me know if this is close to what you're asking and I'll clean it up for you.

Also, if you want to store token validations etc, it's best to put them in the "session" of the return request, this is signed by Play Framework and allows storing data over requests. For larger data you can cache this using the session-id as part of the key.

Read More

Tuesday, July 26, 2016

Amazon and multi customer support in shared multi-tenant model

Leave a Comment

Are there any ready services (by amazon or partners) that help you manage multi-customer aspects of a "pool" [1][2] type service - where all the multi-tenancy is handled by internal context switching, databases are shared, etc.

AWS tools (marketplace, billing manager) seems to be geared toward "provision new service / host by customer" while what I'm looking for is the customer and license management, user association, authentication (including federated authentication integration with multiple customer portals) and perhaps even listing and catalog services - but when a new customer purchase (or change) a license / user / configuration - I expect to get an API call to my already existing solution - in which I'll decide what to do.

Seems like there should be many services like that - but either they are proprietary, or I'm using the wrong keywords to find the information.

[1] http://www.slideshare.net/AmazonWebServices/arc340-multitenant-application-deployment-models/9

[2] https://www.youtube.com/watch?v=DMP0leGZpo4

1 Answers

Answers 1

What you're asking is basically "what tools can I use to build a multi-tenant application".

And the answer will be "it depends". You don't have enough requirements to determine what would be useful or helpful.

However, AWS does have some technologies that may help. Start by looking at AWS Cognito. It can do authentication and data access, and use federated authentication providers. It also handles storing data for multiple users in DynamoDB.

If you're looking for anything more than that, however, you need to provide more information.

Can you shard MySQL across multiple RDS instances, and have one schema per client? Sure, but that's probably not going to work well if you have a million clients - there are limits on the number of schemas per instance.

Same thing with S3 - you can have a bucket per client, or a subdirectory under a shared bucket per client, but I can't tell you which approach is going to work best for your application.

AWS has enough tools to automate the creation of a complete stack per customer - between the API/CLI, CloudFormation, etc - but unless you're dealing with a very high-value product, that's probably not going to be cost effective.

Read More

Monday, April 18, 2016

Query tables across multiple tenants (same table name)

Leave a Comment

I have a system where there is an unknown number of tenants (different database instances on same database server). I have working code where a user logs in and the correct tenant is selected, and I can read the configuration table for that tenant.

I want the application at start time to loop through all tenants, read the configuration and act upon it. Prior to moving to Spring Data JPA (backed by hibernate) this was easy as I was connecting to each database instance separately.

I don't think I can use Spring's @Transactional as it only sets up a single connection.

I hope to use the same repository interface with the same bean, as this works when i only need to hit one tenant at a time.

I do have a class MultiTenantConnectionProviderImpl extends AbstractDataSourceBasedMultiTenantConnectionProviderImpl that will give me a dataSource for a given tenant, but I'm not sure how to use that in a @Service class's method?

2 Answers

Answers 1

I'm not sure if I should remove my previous answer, edit it or what. So if a MOD can let me know proper procedure I'll be happy to comply.

Turns out I was right about the use of @Transactional not going to work. I ended up using an custom implementation of and AbstractRoutingDataSource to replace my MultiTenantConnectionProviderImpl and CurrentTenantResolverImpl. I use this new data source instead of setting the hibernate.multiTenancy hibernate.multi_tenant_connection_provider and hibernate.tenant_identifier_resolver

My temporary override class looks like this:

public class MultitenancyTemporaryOverride implements AutoCloseable {         static final ThreadLocal<String> tenantOverride = new NamedThreadLocal<>("temporaryTenantOverride");      public void setCurrentTenant(String tenantId)     {         tenantOverride.set(tenantId);     }      public String getCurrentTenant()     {         return tenantOverride.get();     }      @Override     public void close() throws Exception     {         tenantOverride.remove();     } } 

My TenantRoutingDataSource looks like this:

@Component public class TenantRoutingDataSource extends AbstractDataSource implements InitializingBean {      @Override     public Connection getConnection() throws SQLException     {         return determineTargetDataSource().getConnection();     }      @Override     public Connection getConnection(String username, String password) throws SQLException     {         return determineTargetDataSource().getConnection(username, password);     }      @Override     public void afterPropertiesSet() throws Exception     {     }      protected String determineCurrentLookupKey()     {         Authentication authentication = SecurityContextHolder.getContext().getAuthentication();         String database = "shared";         if (authentication != null && authentication.getPrincipal() instanceof MyUser)         {             MyUser user = (MyUser) authentication.getPrincipal();             database = user.getTenantId();         }         String temporaryOverride = MultitenancyTemporaryOverride.tenantOverride.get();         if (temporaryOverride != null)         {             database = temporaryOverride;         }         return database;     }      protected DataSource determineTargetDataSource()     {         return selectDataSource(determineCurrentLookupKey());     }      public DataSource selectDataSource(String tenantIdentifier)     {         //I use C3P0 for my connection pool         PooledDataSource pds = C3P0Registry.pooledDataSourceByName(tenantIdentifier);         if (pds == null)             pds = getComboPooledDataSource(tenantIdentifier);         return pds;     }      private ComboPooledDataSource getComboPooledDataSource(String tenantIdentifier)     {         ComboPooledDataSource cpds = new ComboPooledDataSource(tenantIdentifier);         cpds.setJdbcUrl("A JDBC STRING HERE");         cpds.setUser("MyDbUsername");         cpds.setPassword("MyDbPassword");         cpds.setInitialPoolSize(10);         cpds.setMaxConnectionAge(10000);         try         {             cpds.setDriverClass("com.informix.jdbc.IfxDriver");         }         catch (PropertyVetoException e)         {             throw new RuntimeException("Weird error when setting the driver class", e);         }         return cpds;     } } 

Then i just provide my custom data source to my Entity Manager factory bean when creating it.

@Service public class TestService {     public void doSomeGets()     {         List<String> tenants = getListSomehow();         try(MultitenancyTemporaryOverride tempOverride = new MultitenancyTemporaryOverride())         {             for(String tenant : tenants)             {                 tempOverride.setCurrentTenant(tenant);                 //do some work here, which only applies to the tenant             }         }         catch (Exception e)         {             logger.error(e);         }     } } 

Answers 2

I think I'm close to one solution, but I'm not too entirely happy with it. I would love for a better answer to come up.

EDITED: turns out this doesn't quite work, as Spring or Hibernate appears to only call the current tenant identifier resolver once, not for each time a @Transactional method is called

It involves changing the CurrentTenantIdentifierResolver implementation to not only look at the current user (if it is set) to get their current tenant id (up to implementor to figure out how to set that)...it also needs to look at a thread local variable to see if an override has been set.

Using this approach, I can temporarily set the tenantID...call a service method with my multi tenancy transaction manager specified and then get the data.

My Test Service:

@Service public class TestService {     @Transactional(transactionManager = "sharedTxMgr")     public void doSomeGets()     {         List<String> tenants = getListSomehow();         try(MultitenancyTemporaryOverride tempOverride = new MultitenancyTemporaryOverride())         {             for(String tenant : tenants)             {                 tempOverride.setCurrentTenant(tenant);                 doTenantSpecificWork();             }         }         catch (Exception e)         {             logger.error(e);         }     }      @Transactional(transactionManager = "tenantSpecificTxMgr")     public void doTenantSpecificWork()     {         //do some work here, which only applies to the tenant     } } 

My class that wraps setting ThreadLocal, implementing AutoCloseable to help make sure variable is cleaned up

public class MultitenancyTemporaryOverride implements AutoCloseable {     static final ThreadLocal<String> tenantOverride = new ThreadLocal<>();      public void setCurrentTenant(String tenantId)     {         tenantOverride.set(tenantId);     }      public String getCurrentTenant()     {         return tenantOverride.get();     }      @Override     public void close() throws Exception     {         tenantOverride.remove();     }  } 

My tenant resolver implementation that uses the thread local

public class CurrentTenantIdentifierResolverImpl implements CurrentTenantIdentifierResolver {      @Override     public String resolveCurrentTenantIdentifier()     {         Authentication authentication = SecurityContextHolder.getContext().getAuthentication();         logger.debug(ToStringBuilder.reflectionToString(authentication));         String database = "shared";         if (authentication != null && authentication.getPrincipal() instanceof MyUser)         {             MyUser user = (MyUser) authentication.getPrincipal();             database = user.getTenantId();         }         String temporaryOverride = MultitenancyTemporaryOverride.tenantOverride.get();         if(temporaryOverride != null)         {             database = temporaryOverride;         }         return database;     } 
Read More