Rate Limiting Access to the Reddit API

1. Overview

In this quick article we're going to continue improving our small Reddit app by rate limiting the way it has access to the live Reddit API.

The simple idea is that we want to make sure we don't hit their API to much – otherwise Reddit will start blocking the requests. We're going to make good use of the Guava RateLimiter to get there.

2. A Custom RedditTemplate

First, let's create a Reddit template – a small client for the Reddit API – which will consolidate all low level communication to a single component:

@Component
@Scope(value = "session", proxyMode = ScopedProxyMode.TARGET_CLASS)
public class RedditTemplate {

    @Autowired
    @Qualifier("redditRestTemplate")
    private OAuth2RestTemplate redditRestTemplate;

    private RateLimiter rateLimiter;

    public RedditTemplate() {
        rateLimiter = RateLimiter.create(1);
    }
    
    public JsonNode getUserInfo() {
        rateLimiter.acquire();
        return redditRestTemplate.getForObject(
          "https://oauth.reddit.com/api/v1/me", JsonNode.class);
    }
    
    public JsonNode submitPost(MultiValueMap<String, String> params) {
        rateLimiter.acquire();
        return redditRestTemplate.postForObject(
          "https://oauth.reddit.com/api/submit", params, JsonNode.class);
    }
    
    public String needsCaptcha() {
        rateLimiter.acquire();
        return redditRestTemplate.getForObject(
          "https://oauth.reddit.com/api/needs_captcha.json", String.class);
    }
    
    public String getNewCaptcha() {
        rateLimiter.acquire();
        Map<String, String> param = new HashMap<String, String>();
        param.put("api_type", "json");
        return redditRestTemplate.postForObject(
          "https://oauth.reddit.com/api/new_captcha", param, String.class, param);
    }
    
    public OAuth2AccessToken getAccessToken() {
        rateLimiter.acquire();
        return redditRestTemplate.getAccessToken();
    }
}

A few interesting things are happening here.

First – we're using the Session scope for this bean – simply so that each user/session in our app will get its own RedditTemplate instance.

Now – the OAuth2RestTemplate already has support for keeping credentials session scoped, but we're going beyond that here and making the actual bean instance session scoped – so that we can also rate limit each user separately.

Which leads us to the actual rate limiting logic – simply put, we're using the Guava RateLimiter to acquire a permit before letting the request through and hitting the live API.

3. The RedditController

Next – let's start using this new RedditTemplate in the RedditContoller – for example:

@Controller
public class RedditController {
    @Autowired
    private RedditTemplate redditTemplate;

    @Autowired
    private UserRepository userReopsitory;

    @RequestMapping("/login")
    public String redditLogin() {
        JsonNode node = redditTemplate.getUserInfo();
        
        loadAuthentication(node.get("name").asText(), 
          redditTemplate.getAccessToken());
        return "redirect:home.html";
    }
}

4. Conclusion

In this part of the Case Study we added rate limiting to the Reddit application, to make sure we're not blocked by the live API for to much activity.

This isn't a theoretical problem either – but actually something that I ran into a couple of times using the app.

It's this kinds of small improvements that are eventually going to lead to a mature and usable application – so I'm excited about this particular step.