Tuesday, May 2, 2017

Is it safe to store my 'next' url in a signed cookie and redirect to it carefree?

Leave a Comment

I'm using Flask and it's occurred to me it could be a rather elegant solution to redirect back to the user's last page after login/logout by simply placing a session['next'] = request.url at each endpoint of my application and to just have my login/logout functions redirect right to session.get('next'). This is even similar to an option in the Flask-Login extension if you enable USE_SESSION_FOR_NEXT.

I would like to confirm this is a safe workflow but am not security-savvy to recognize if there are any ways to spoof the request.url or if I should still be validating the next url prior to redirecting, as is specified here:

http://flask.pocoo.org/snippets/62/

Is there a reason this method is not more commonly deployed? It seems like a nice, clean, easy solution that keeps URL's clean, minimizes for fields/processing, and removes a vulnerability to open redirect attacks if you are not taking the extra steps to validate the next url. What's the catch?

4 Answers

Answers 1

TL;DR

  1. Yes, it is safe.
  2. Don't worry about hiding the url, or preventing "spoofing". In reality, you can do neither, so you prepare for both.
  3. Potential bug: once you set the redirect on the session, the next login will follow it no matter what, until the session expires. A way to remedy this is just save the redirect path as a GET parameter instead (see bottom of answer).
  4. Here is a nice cheat sheet for redirect and forwarding security concerns.

This is a safe workflow, assuming your server authenticates incoming requests to all secure resources.

When you ask "if I should still be validating the next url prior to redirecting", the answer to that is no, but you need to validate all requests to that url (after redirecting) to make sure they are logged in.

In your question, it sounds like you are trying to keep the url hidden from the user until they are logged in. That shouldn't be necessary.

For example, let's say you have two urls:

url 1:  "/login"  # anyone can access this url 2:  "/secure_page"  # only a logged in user can access this 

Let's say a user is not logged in, and tries to navigate to yoursite.com/secure_page. They should be redirected to your login page every time. If they know the secure_page url, that shouldn't compromise your security at all. In fact, they must already know that url because they navigated to that page in the first place, so either they typed it in, clicked a link, or it was saved in a bookmark or a browsing history. The important thing is that when you handle requests to secure_page, you require them to be logged in.

You ask if they can "spoof" the url that they are redirected to. They cannot, when you save it in the session like that. However, they can hit any url they want after they log in, so it doesn't matter if they "spoof" that url.

So since they already know that url and they can hit any url they want after they log in, you don't need to keep that url secret from the user. The only advantage of doing that is an aesthetically cleaner url. This is why you see many login pages that look like this:

http://yoursite.com/login?next=secure_page 

What is happening there is they are saving that next url as an HTTP GET parameter. While it is less "clean", it is more explicit, so it has its pros and cons. And if they were to revisit the login page later, that redirect would no longer occur, which may be your desired behavior. With your current code, once that redirect is on the session, this next login after that will follow the redirect until the session expires, which could be after the user walks away and someone else uses that web browser.

Many sites do it the way I showed above. Django does it that way. In this case, a malicious link could provide the "next" url and have it redirect to some phishing site, but that could be easily prevented by ensuring the "next" url doesn't navigate away from your domain.

Here is a basic cheat sheet for redirect/forward security (you are redirecting, but others landing here may be considering forwarding requests).

Answers 2

To be honest, I can't think of any practical way to exploit this. Rather, I can't think of any practical way to exploit this, assuming you are properly validating incoming requests on other pages, and have properly implemented authentication and access control. That being said, I know nothing about your application, and there are certainly attackers out there a lot brighter than me, so take that with a grain of salt.

That being said, I don't see any reason why you shouldn't validate the URL, at the very least to make sure you're redirecting to a page on the same domain. The performance cost is trivial, and I'm of the opinion that anything related to the login/logout flow deserves an extra level of scrutiny. I would also make sure that you have an acceptable default case for when you receive a request without this field in the cookie.

Answers 3

As @brendan commented, this solution will work fine, use a @login_required decorator to protect the view.

Two cases:

Loggin in

You are at /index (non-protected view) >> login successfully to a @login_protected view, in this case, if you go back, as /index is accessible for everybody, you really do not care

Logging out

You are at /profile view (protected view) >> successful logout to a non-protected view, if you go back, the decorator will not allow you to enter the view

Answers 4

There is one small issue though. If you have multiple frames in your html there is an exploit and is called as clickjacking. You will need to set the following headers so that browser takes care of the remaining.

<X-Frame-Options','DENY'> 

Though it does not concern redirection but it can be used here. Be careful.

Read More

Debug loading issue

Leave a Comment

I'm having trouble debugging an extremely strange bug.

It happens rarely, and at seemingly random places on the page.

The HTML will stop, and start loading the page again - see screenshot below:

enter image description here

If I reload the page then 99% of the time it works fine. We're using Magento on an nginx server but the issue also happens on my local dev box.

There are no errors generated when this happens that I can see (checked nginx, php-fpm, mysql and Magento logs).

Does anyone have any ideas on how I could debug this issue?

0 Answers

Read More

Monday, May 1, 2017

Python Enum shows weird behavior when using same dictionary for member values

Leave a Comment

I don't understand why this Enum doesn't have all the members I defined, when I assign a dict as each member's value:

from enum import Enum  class Token(Enum):     facebook = {     'access_period': 0,     'plan_name': ''}      instagram = {     'access_period': 0,     'plan_name': ''}      twitter = {     'access_period': 0,     'plan_name': ''}  if __name__ == "__main__":     print(list(Token)) 

The output is:

[<Token.twitter: {'plan_name': '', 'access_period': 0}>] 

… but I expected something like:

[<Token.facebook:  {'plan_name': '', 'access_period': 0}>,  <Token.instagram: {'plan_name': '', 'access_period': 0}>,  <Token.twitter:   {'plan_name': '', 'access_period': 0}>] 

Why aren't all the members shown?

2 Answers

Answers 1

Enum enforces unique values for the members. Member definitions with the same value as other definitions will be treated as aliases.

Demonstration:

Token.__members__ # OrderedDict([('twitter', #               <Token.twitter: {'plan_name': '', 'access_period': 0}>), #              ('facebook', #               <Token.twitter: {'plan_name': '', 'access_period': 0}>), #              ('instagram', #               <Token.twitter: {'plan_name': '', 'access_period': 0}>)])  assert Token.instagram == Token.twitter 

The defined names do all exist, however they are all mapped to the same member.

Have a look at the source code if you are interested:

# [...] # If another member with the same value was already defined, the # new member becomes an alias to the existing one. for name, canonical_member in enum_class._member_map_.items():     if canonical_member._value_ == enum_member._value_:         enum_member = canonical_member         break else:     # Aliases don't appear in member names (only in __members__).     enum_class._member_names_.append(member_name) # performance boost for any member that would not shadow # a DynamicClassAttribute if member_name not in base_attributes:     setattr(enum_class, member_name, enum_member) # now add to _member_map_ enum_class._member_map_[member_name] = enum_member try:     # This may fail if value is not hashable. We can't add the value     # to the map, and by-value lookups for this value will be     # linear.     enum_class._value2member_map_[value] = enum_member except TypeError:     pass # [...] 

Further, it seems to me that you want to exploit the Enum class to modify the value (the dictionary) during run-time. This is strongly discouraged and also very unintuitive for other people reading/using your code. An enum is expected to be made of constants.

Answers 2

As @MichaelHoff noted, the behavior of Enum is to consider names with the same values to be aliases1.

You can get around this by using the Advanced Enum2 library:

from aenum import Enum, NoAlias  class Token(Enum):     _settings_ = NoAlias     facebook = {         'access_period': 0,         'plan_name': '',         }      instagram = {         'access_period': 0,         'plan_name': '',         }      twitter = {         'access_period': 0,         'plan_name': '',         }  if __name__ == "__main__":     print list(Token) 

Output is now:

[   <Token.twitter: {'plan_name': '', 'access_period': 0}>,   <Token.facebook: {'plan_name': '', 'access_period': 0}>,   <Token.instagram: {'plan_name': '', 'access_period': 0}>,   ] 

To reinforce what Michael said: Enum members are meant to be constants -- you shouldn't use non-constant values unless you really know what you are doing.


A better example of using NoAlias:

class CardNumber(Enum):      _order_ = 'EIGHT NINE TEN JACK QUEEN KING ACE'  # only needed for Python 2.x     _settings_ = NoAlias      EIGHT    = 8     NINE     = 9     TEN      = 10     JACK     = 10     QUEEN    = 10     KING     = 10     ACE      = 11 

1 See this answer for the standard Enum usage.

2 Disclosure: I am the author of the Python stdlib Enum, the enum34 backport, and the Advanced Enumeration (aenum) library.

Read More

How to make future calls and wait until complete with Python?

Leave a Comment

I have the following code where I have a list of usernames and I try and check if the users are in a specific Windows Usergroup using net user \domain | find somegroup.

The problem is that I run that command for about 8 usergroups per username and it is slow. I would like to send off these calls using futures and even separate threads (if it makes it quicker).

I just have to wait at the end before i do anything else. How do I go about doing it in Python?

for one_username in user_list:     response = requests.get(somecontent)      bs_parsed = BeautifulSoup(response.content, 'html.parser')      find_all2 = bs_parsed.find("div", {"class": "QuickLinks"})     name = re.sub("\s\s+", ' ', find_all2.find("td", text="Name").find_next_sibling("td").text)      find_all = bs_parsed.find_all("div", {"class": "visible"})     all_perms = ""     d.setdefault(one_username + " (" + name + ")", [])     for value in find_all:         test = value.find("a", {"onmouseover": True})         if test is not None:             if "MyAppID" in test.text:                 d[one_username + " (" + name + ")"].append(test.text)      for group in groups:         try:             d[one_username + " (" + name + ")"].append(check_output("net user /domain " + one_username + "| find \"" + group + "\"", shell=True, stderr=subprocess.STDOUT).strip().decode("utf-8"))         except Exception:             pass 

3 Answers

Answers 1

(This answer currently ignores HTML parsing your code does ... you can queue that into a pool identically to how this approach queues the net user calls)

First, lets define a function that takes a tuple of (user, group) and returns the desired information.

# a function that calls net user to find info on a (user, group) def get_group_info(usr_grp):     # unpack the arguments     usr, grp = usr_grp      try:         return (usr, grp,                  check_output(                     "net user /domain " + usr + "| find \"" + grp + "\"",                      shell=True,                      stderr=subprocess.STDOUT                     ).strip().decode("utf-8")))     except Exception:         return (usr, grp, None) 

Now, we can run this in a thread pool using multiprocessing.dummy.Pool

from multiprocessing.dummy import Pool import itertools  # create a pool with four worker threads pool = Pool(4)  # run get_group_info for every user, group async_result = pool.map_async(get_group_info, itertools.product(user_list, groups))  # now do some other work we care about ...  # and then wait on our results results = async_result.get() 

The results are a list of (user, group, data) tuples and can be processed as you desire.

Note: This code is currently untested due to a difference in platforms

Answers 2

It seems like producer consumer problem.

The main thread should generate the tasks

class Task:     def Task(self,user,group)         self.user  = user         self.group = group     def run(self):         pass # call command with self.user and self.group and process results  twp = TaskWorkerPool(4) for group in groups:     twp.add( Task(user,group) ) twp.wait() 

Answers 3

In python 3, a more simple and convenient solution is to use concurrent.futures.

The concurrent.futures module provides a high-level interface for asynchronously executing callables. Reference...

import concurrent.futures   # Get a list containing all groups of a user def get_groups(username):     # Do the request and check here     # And return the groups of current user with a list     return list()  with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:     # Mark each future with its groups     future_to_groups = {executor.submit(get_groups, user): user                         for user in user_list}      # Now it comes to the result of each user     for future in concurrent.futures.as_completed(future_to_groups):         user = future_to_groups[future]         try:             # Receive the returned result of current user             groups = future.result()         except Exception as exc:             print('%r generated an exception: %s' % (user, exc))         else:             # Here you do anything you need on `groups`             # Output or collect them             print('%r is in %d groups' % (user, len(groups))) 

See here where this example comes from.

EDIT:

If you need to do each check in seperate thread:

import concurrent.futures   # Check if a `user` is in a `group` def check(user, group):     # Do the check here     # And return True if user is in this group, False if not     return True  with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:     # Mark each future with its user and group     future_to_checks = {executor.submit(check, user, group): (user, group)                         for user in user_list for group in group_list}      # Now it comes to the result of each check     # The try-except-else clause is omitted here     for future in concurrent.futures.as_completed(future_to_checks):         user, group = future_to_checks[future]         in_group = future.result()         if in_group is True:             print('%r is in %r' % (user, group)) 
Read More

Strange behavior with taskScheduler pool

Leave a Comment

I have two spring boot app (1.4.3.RELEASE) which are on the same server. The app A is a monolithic app which contains a part of code used to process alerts and the app B is a new dedicated app which only process alerts. The goal here is to break the monolotic app in small apps. For now, the two codes run together because I have old systems which always call the app A.

The two app have a taskScheduler configured based on a ThreadPoolTaskScheduler.

@Configuration public class TaskSchedulerConfig {      @Bean     public TaskScheduler taskScheduler() {         ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler();         threadPoolTaskScheduler.setWaitForTasksToCompleteOnShutdown(true);         threadPoolTaskScheduler.setPoolSize(100);          return threadPoolTaskScheduler;     } } 

Yesterday, I have experienced a strange behavior :

  1. An alert has been detected and sent to the new app B -> OK
  2. The app B received the alert and start to process it based on the taskScheduler -> OK
  3. The first step has been processed by the app B -> OK
  4. The second step has been processed by the app A -> NOK, strange behavior
  5. The third step has been processed by the app B as expected -> OK

How can this be possible? For me, each taskScheduler is attached to the app which created it. Where am I wrong?

UPDATE

I have a real box which emit alerts. Those alerts must be processed by a new application. But I have also old box which have not migrate to the new system. So I have the processing code in two different projects.

I have a new box with the new code which have created an alert on the new system. This alert generate a state machine which is processed in async with a task scheduler. After alert creation, the new app starts to process the state machine and at the middle of the processing the old application wake up and process a step of the alert. After that, the new application wake up again and normally close the alert.

The problem is : why the old application wake up to process an alert? Is there a known issue with a threadPoolTaskScheduler?

1 Answers

Answers 1

There is no way that two different applications have this behavior since they are running in isolated processes. Threads (of the same process) run in a shared memory space, while processes run in separate memory spaces, so there is no 'bridge' between them.

If they share the same database, they might be listening to the same events, but only if you have that logic implemented by you.

If I had to guess, given that both are webapps I'd say that there might be some HTTP call somewhere in the code, still aiming to an old endpoint, or some other trigger (crons?) inside the server that is kickstarting the old app.

Read More

Handling authentication in Nodejs with passport-facebook-token, request coming from frontend Facebook SDK

Leave a Comment

I am working on a Unity App. For login, there are two methods, one using Email and another using Facebook. In case of login separately, I do not have any problem. Registration and Login with Email works perfectly. And Login with Facebook works perfectly as well. Here's the workflow, I created just to make you clear. Login work flow

tl;dr [read update]

There's another schema for account, which is used for login.

var Account = new Schema({   email: String,   password: String,   facebookId: String }); 

Things to know about the backend API.

  1. Passport is used for Authentication
  2. Successful login returns email and token to the client through API.
  3. On client, token is most to play game and use the overall features.

As I said, I have already covered the part when if a client registers and login using email, then client can use the app. But my confusion is handling the logins with Facebook. Facebook SDK is already integrated with the Unity App, and Login is success.

Now, how can I use the Facebook login information that is generated by the Facebook SDK onto my back end, so that I can authorize the user throughout the system, as done in email login.

Going through other questions in SO and Google, I came across passport-facebook-token, I also tried using the plugin but could not came up with the logic and flow for handling the data from SDK into the Nodejs API. Can someone me help understand how it is done?

Update 1: Using passport-facebook-token

Strategy on index.js

passport.use(new FacebookTokenStrategy({     clientID: FACEBOOK_APP_ID,     clientSecret: FACEBOOK_APP_SECRET   }, function(accessToken, refreshToken, profile, done) {     Account.findOrCreate({facebookId: profile.id}, function (error, user) {       return done(error, user);     });   } )); 

Controller API

api.post('/auth/facebook/token', passport.authenticate('facebook-token'), function (req, res) {   console.log(req.user);   // do something with req.user   res.sendStatus(req.user? 200 : 401); } ); 

Now, there is no error shown, but the data is not inserted into Account Schema, I have this findOrCreate() function in Model.

Account.statics.findOrCreate = function findOrCreate(profile, cb){ var userObj = new this(); this.findOne({facebookId : profile.id},function(err,result){     if(!result){         userObj.facebookId = profile.id;         //....         userObj.save(cb);     }else{         cb(err,result);     } }); }; 

1 Answers

Answers 1

you can use facebook-passport for that, you can check the documentation here: https://github.com/jaredhanson/passport-facebook but basically, after you have already set up your developer account and got your keys from the developer site of facebook you can implement a FacebookStrategy object like following where you have to specify your credential and also a callback that in the documentation example is an http request to another resource of an express server where you can then save the data to mongo

passport.use(new FacebookStrategy({     clientID: FACEBOOK_APP_ID,     clientSecret: FACEBOOK_APP_SECRET,     callbackURL: "http://localhost:3000/auth/facebook/callback" }, function(accessToken, refreshToken, profile, cb) {     User.findOrCreate({ facebookId: profile.id }, function (err, user) {     return cb(err, user);    });  }  )); 
Read More

How to fill QTextEdit of another class

1 comment

I have two classes.

The class mask_n_functions controls a form that fills up a SQLite database. There is the method call_inventory_mask which opens a new mask class inventory_function(QDialog, Ui_Inventory):. There I can check some CheckBoxes and fill up a list (InventoryList) with some elements.

Now I want to fill up a QTextEdit of a form which is called with the class mask_n_functions with resulting InventoryListString.

I want to use this for a QGIS Plugin. I use Notepad++ for coding.

I tried self.mask_n_functions.inventory_list.setText(InventoryListString),

UPDATE: There is no python error occurring within QGIS. The QTextEdit field of the mask, called with the class mask_n_functions, still remains empty.

Here is my (reduced) code:

class inventory_function(QDialog, Ui_Inventory):      def __init__(self, parent):         QDialog.__init__(self, parent)         self.setupUi(self)          self.get_inv.clicked.connect(self.getInventory)         self.close_inv.clicked.connect(self.closeInventory)      def getInventory(self):         getInventoryList = [['inv1','Example2'],['inv2','Example2'],['inv3','Example3']]         InventoryList = []         for l in getInventoryList:             checkboxstring = str(l[0])             checkboxname = checkboxstring.strip()             checkbox = getattr(self, checkboxname)             if checkbox.isChecked():                 InventoryList.append(l[1])         InventoryListString = '%s' % ', '.join(map(str, InventoryList))         self.mask_n_functions.inventory_list.setText(InventoryListString) # inventory_list is a QTextEdit() widget of Ui_MainForm in the class mask_n_functions         self.close()      def closeInventory(self):         self.close()  class mask_n_functions(QDialog, Ui_MainForm):      def __init__(self, parent):         QDialog.__init__(self, parent)         self.setupUi(self)          global now         now = datetime.datetime.now()          global username         username = getpass.getuser()          ...          self.choose_inventory.clicked.connect(self.call_inventory_mask)      def call_inventory_mask(self):         inventory_mask = inventory_function(self)         inventory_mask.show() 

0 Answers

Read More