HTTP API – downloading webpages using web requests

2021/01 02 14:01

When you’re maintaining scoreboards or other such things that require regular HTTP request access to servers, you can use the HTTP API to perform such web request tasks.

Getting ready

Have a server that you’re allowed to request data via HTTP from. You can use a public server of any type to try out HTTP requests, if you’d like.

How to do it…

  1. Link to the HTTP API in your ProjectName.Build.cs file:
using UnrealBuildTool;

public class Chapter_11 : ModuleRules
{
public Chapter_11(ReadOnlyTargetRules Target) : base(Target)
{
PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;

PublicDependencyModuleNames.AddRange(new string[] {
"Core", "CoreUObject", "Engine", "InputCore" });
PublicDependencyModuleNames.AddRange(new string[] {
"GameplayAbilities", "GameplayTags", "GameplayTasks" });
PublicDependencyModuleNames.AddRange(new string[] {
"HTTP" });


PrivateDependencyModuleNames.AddRange(new string[] { });

// Uncomment if you are using Slate UI
// PrivateDependencyModuleNames.AddRange(new string[]
// { "Slate", "SlateCore" });

// Uncomment if you are using online features
// PrivateDependencyModuleNames.Add("OnlineSubsystem");

// To include OnlineSubsystemSteam, add it to the plugins section in your uproject file with the Enabled attribute set to true
}
}
  1. In the file that you will send your web request from (in my case, I’ll be using the Chapter_11GameModeBase class), include the new additions to the following code snippet:
#pragma once

#include "CoreMinimal.h"
#include "GameFramework/GameModeBase.h"
#include "Runtime/Online/HTTP/Public/HttpManager.h"
#include "Runtime/Online/HTTP/Public/HttpModule.h"
#include "Runtime/Online/HTTP/Public/HttpRetrySystem.h"
#include "Runtime/Online/HTTP/Public/Interfaces/IHttpResponse.h"
using namespace FHttpRetrySystem;
#include "Chapter_11GameModeBase.generated.h"
  1. Construct an IHttpRequest object from FHttpModule using the following code:
TSharedRef<IHttpRequest> 
http=FHttpModule::Get().CreateRequest();

FHttpModule is a singleton object. One copy of it exists for the entire program that you are meant to use for all interactions with the FHttpModule class.

  1. Attach your function to run to the IHttpRequest object’s FHttpRequestCompleteDelegate, which has the following signature:
void HttpRequestComplete( FHttpRequestPtr request,
FHttpResponsePtr response, bool success );
  1. The delegate is found inside of the IHttpRequest object as http->OnProcessRequestComplete():
FHttpRequestCompleteDelegate& delegate = http-
>OnProcessRequestComplete();

There are a few ways to attach a callback function to the delegate. You can use the following methods:

  • A lambda using delegate.BindLambda():
delegate.BindLambda( 
  // Anonymous, inlined code function (aka lambda) 
  []( FHttpRequestPtr request, FHttpResponsePtr response, bool 
success ) -> void { UE_LOG( LogTemp, Warning, TEXT( "Http Response: %d, %s" ), request->GetResponse()->GetResponseCode(), *request->GetResponse()->GetContentAsString() ); });
  1. Specify the URL of the site you’d like to hit:
http->SetURL( TEXT( "http://unrealengine.com" ) ); 
  1. Process the request by calling ProcessRequest:
http->ProcessRequest();

Then, when you run this code, you should notice the contents of the URL you pointed to being displayed:

HTML content of unrealengine.com displayed

Of course, in this instance, it’s a web page, but you can easily point this to a CSV file, text document, or anything to obtain information for your project!

How it works…

The HTTP object is all you need to send off HTTP requests to a server and get HTTP responses. You can use the HTTP request/response for anything that you wish; for example, submitting scores to a high scores table or to retrieve text to display in-game from a server.

They are decked out with a URL to visit and a function callback to run when the request is complete. Finally, they are sent off via FManager. When the web server responds, your callback is called and the results of your HTTP response can be shown.

There’s more…

There are also other ways in which you can attach a callback function to the delegate:

  • Using any UObject’s member function:
delegate.BindUObject(this, &AChapter_11GameModeBase::HttpRequestComplete);

You cannot attach to UFunction directly here as the .BindUFunction() command requests arguments that are all UCLASS, USTRUCT, or UENUM.

  • With any plain old C++ object’s member function, using .BindRaw:
PlainObject* plainObject = new PlainObject();
delegate.BindRaw( plainObject, &PlainObject::httpHandler );
// plainObject cannot be DELETED Until httpHandler gets
// called..

You have to ensure that your plainObject refers to a valid object in memory at the time that the HTTP request completes. This means that you cannot use TAutoPtr on plainObject, because that will deallocate plainObject at the end of the block in which it is declared, but that may be before the HTTP request completes.

  • Using a global C-style static function:
// C-style function for handling the HTTP response: 
void httpHandler( FHttpRequestPtr request,  
FHttpResponsePtr response, bool success ) 
{ 
  Info( "static: Http req handled" ); 
} 
delegate.BindStatic( &httpHandler ); 

When using a delegate callback with an object, be sure that the object instance that you’re calling back on lives on at least until the point at which the HttpResponse arrives back from the server. Processing the HttpRequest takes real time to run. It is a web request after all—think of waiting for a web page to load.

You have to be sure that the object instance on which you’re calling the callback function has not deallocated on you between the time of the initial call and the invocation of your HttpHandler function. The object must still be in memory when the callback returns after the HTTP request completes.

You cannot simply expect that the HttpResponse function happens immediately after you attach the callback function and call ProcessRequest()! Using a reference counted UObject instance to attach the HttpHandler member function is a good idea to ensure that the object stays in memory until the HTTP request completes.

You can see an example of all seven possible ways it can be used inside of the Chapter_11GameModeBase.cpp file in the Example Code for this chapter.

You can set additional HTTP request parameters via the following member functions:

  • SetVerb(), to change whether you are using the GET or POST method in your HTTP request
  • SetHeaders(), to modify any general header settings you would like