Capture text from screen in C++ application

This article explains how to create a simple Visual C++ application which captures text from screen using TextGRAB SDK. We will create application similar to standard C++ demo which comes with SDK installer package – a simple dialog based application. It will allow capturing text snapshot and text from screen (capture text from rectangle, capture word from point, capture string).

Prerequisites
1. Creating your C++ project
2. User interface design
3. TextGRAB SDK COM Object import
4. SDK Initialization
5. Text capture demo dialog member methods implementation
6. Selection handler and text capture functionality implementation
Conclusion

Prerequisites

TextGRAB SDK 3.1 should be installed.

1. Creating your C++ project

Launch Microsoft Visual Studio.
Select “File\New\Project” menu.
In the list of templates select “MFC Application”.
Type Project name “demo”
Press OK
Go to “Application Type” tab and select “Dialog based” option.
Press Finish.

2. User interface design

Open IDD_DEMO_DIALOG in dialog editor and place controls as shown on screenshot:

a2_cs_form

Or alternatively you can change code in demo.rc file as follows:

IDD_DEMO_DIALOG DIALOGEX 0, 0, 390, 262
STYLE DS_SETFONT | DS_MODALFRAME | DS_FIXEDSYS | WS_POPUP | WS_VISIBLE | WS_CAPTION | WS_SYSMENU
EXSTYLE WS_EX_APPWINDOW
CAPTION "TextGRAB SDK Demo v3.1"
FONT 8, "MS Shell Dlg", 0, 0, 0x1
BEGIN
    CONTROL        "&Enable text capturing (Use mouse with Ctrl-Shift to select the rectangle in window)"
,IDC_ENABLE_CAPTURE,
                    "Button",BS_AUTOCHECKBOX | WS_TABSTOP,7,13,279,10
    PUSHBUTTON      "Select&&Capture",IDC_SELECTANDCAPTURE,287,13,64,14,WS_DISABLED
    LTEXT           "Capture method:",IDC_STATIC,8,36,81,13
    COMBOBOX        IDC_COMBO_CAPTURE_METHOD,7,50,241,30,CBS_DROPDOWNLIST|CBS_SORT|WS_VSCROLL|WS_TABSTOP
    CONTROL         "",IDC_TXTSNAPSHOT,"SysListView32",LVS_REPORT | LVS_ALIGNLEFT | NOT WS_VISIBLE | 
WS_BORDER | WS_TABSTOP,7,81,344,187
    EDITTEXT        IDC_TEXT,7,81,344,187,ES_MULTILINE | ES_AUTOHSCROLL | WS_VSCROLL
    LTEXT           "&Hardware ID",IDC_STATIC,287,38,40,8
    EDITTEXT        IDC_EDIT_HWID,284,49,67,14,ES_AUTOHSCROLL
END

Please note that there is hidden list view control which we will later use for displaying text snapshot capturing results. Anyway, if you will change demo.rc manually – do not forget to add corresponding code to resource.h, e.g.:

#define IDC_ENABLE_CAPTURE			1001
#define IDC_SELECTANDCAPTURE			1002
#define IDC_COMBO_CAPTURE_METHOD		1003
#define IDC_TXTSNAPSHOT			1004
#define IDC_TEXT				1005
#define IDC_EDIT_HWID				1006

When this is done – for the convenience let us add some DDX and some handlers to message map. Let us start with adding some public members to CdemoDlg (in demoDlg.h file):

	CComboBox m_ComboCaptureMethod;  // Capture method combo box.
	CListCtrl m_lvTextSnasphot;  // Text snapshot list view.
	CEdit m_Text;  // Control with captured text.
	CEdit m_txtHardwareID;  // Control displaying hardware id.
 
	afx_msg void OnClose();  // Application closes.
	afx_msg void OnBnClickedSelAndCap();  // Click on "Select & Capture" button.
	afx_msg void OnBnClickedEnableCapture();  // Click on "Enable text capture" text box.
	afx_msg void OnCbnSelchangeComboCapMethod();  // Capture method changes.
	afx_msg LRESULT OnSelection(WPARAM, LPARAM);  // Selection finished.

First group of members represent some controls on the dialog while second represents event handlers for controls.
Now we need to change CdemoDlg::DoDataExchange in the following way:

void CdemoDlg::DoDataExchange(CDataExchange* pDX)
{
	CDialog::DoDataExchange(pDX);
	DDX_Control(pDX, IDC_COMBO_CAPTURE_METHOD, m_ComboCaptureMethod);
	DDX_Control(pDX, IDC_TXTSNAPSHOT, m_lvTextSnasphot);
	DDX_Control(pDX, IDC_TEXT, m_Text);
	DDX_Control(pDX, IDC_EDIT_HWID, m_txtHardwareID);
}

Before adding new entries to dialog message map – we need to define TextGRAB SDK selection message at the top of demoDlg.cpp file like this:

#define WM_SELECTION     WM_USER + 1024

WM_SELECTION method is sent to a window when user finished selecting some area on screen with TextGRAB SDK internal selection tool. Now we are ready to add new entries to a message map:

	ON_WM_CLOSE()
	ON_BN_CLICKED(IDC_SELECTANDCAPTURE, &CdemoDlg::OnBnClickedSelAndCap)
	ON_BN_CLICKED(IDC_ENABLE_CAPTURE, &CdemoDlg::OnBnClickedEnableCapture)
	ON_CBN_SELCHANGE(IDC_COMBO_CAPTURE_METHOD, &CdemoDlg::OnCbnSelchangeComboCapMethod)
	ON_MESSAGE(WM_SELECTION, &CdemoDlg::OnSelection)

3. TextGRAB SDK COM Object import

Now we need to import TextGRAB SDK COM object into our project. For this purpose – open demoDlg.h file and add following code at the top of the file:

#import "C:\\Program Files\\TextGRAB SDK\\tgsdkx.dll" rename_namespace("TGSDK")

After this will be done – rebuild the project so it will create object wrapper and auto-completion will be available for TextGRAB SDK methods and properties.

4. SDK Initialization

Add smart initializer class (CoInitialize, CoUninitialize) to demo.cpp:

class SmartCoInititialize{
public:
    SmartCoInititialize() { CoInitialize( NULL ); }
    ~SmartCoInititialize() { CoUninitialize(); }
};

and add following code at the very beginning of the InitInstance method:

BOOL CdemoApp::InitInstance()
{
    SmartCoInititialize coInit;
    // ...

Add following protected members to CdemoDlg ( in demoDlg.h file):

CComPtr< TGSDK::ITextGRABSDK > m_spITextGRABSDK;  // TextGARB SDK Interface smart pointer.
BOOL m_bSDKInitialized;  // Indicates if SDK was initialized or not.

Modify demoDlg constructor in demoDlg.cpp file as it is shown here:

CdemoDlg::CdemoDlg(CWnd* pParent /*=NULL*/)
    : CDialog(CdemoDlg::IDD, pParent)
{
    m_hIcon = AfxGetApp()->LoadIcon(IDR_MAINFRAME);
 
    m_bSDKInitialized = FALSE;  // SDK is not initialized yet.
 
    // Create SDK instance.
    HRESULT hr = m_spITextGRABSDK.CreateInstance( __uuidof( TGSDK::TextGRABSDK ) );   
 
    // Check result.
    if( SUCCEEDED( hr ) )
    {
        m_spITextGRABSDK->SetLicense( L"", L"" );  // Set license.
        m_bSDKInitialized = TRUE;  // Indicate that SDK was initialized.
    }
    else
    {
        // Show error message.
        MessageBox( _T( "Can't create COM component" ), _T( "Error" ), MB_ICONERROR | MB_OK );
    }
}

Also it is necessary to add the following code at the end of CdemoDlg::OnInitDialog() method:

    if( m_bSDKInitialized )  // Check if SDK was initialized.
    {           
        // Register dialog window as selection notification receiver.
        m_spITextGRABSDK->RegisterSelectionNotify( ( INT_PTR ) m_hWnd );
 
        // Get the hardware id and display it on the dialog.
        CComBSTR bstrHWID;
        if ( SUCCEEDED( m_spITextGRABSDK->GetHardwareID( &bstrHWID ) ) ){
            m_txtHardwareID.SetWindowText(bstrHWID);
        }
 
        // Get trial period information and display it in the title.
        LONG dwTotal = 0;
        LONG dwLeft = 0;
        if( SUCCEEDED( m_spITextGRABSDK->GetTrialInfo( &dwTotal, &dwLeft ) ) ){                   
            // Format trial period information .
            CString strCaption, strPeriodInfo;
            strPeriodInfo.Format( _T( "  Trial, %ld day(s) left from %ld" ), dwLeft, dwTotal );
 
            // Get current window caption, append trial period info and set as a new title.           
            this->GetWindowText( strCaption );
            strCaption.Append( strPeriodInfo );
            this->SetWindowText( strCaption );
        }
 
        // Add selection methods to combo box.
        m_ComboCaptureMethod.AddString(_T("Text in selected rectangle"));
        m_ComboCaptureMethod.AddString(_T("Text string at selection (Just click on text in window)"));
        m_ComboCaptureMethod.AddString(_T("Text snapshot of window (Just click on window)"));
        m_ComboCaptureMethod.AddString(_T("Whole window text capturing (Just click in window)"));
        m_ComboCaptureMethod.SetCurSel(0);
 
        // Add column names to text snapshot list view.
        TCHAR* szColNames[] = {
            _T("Screen Rectangle"),
            _T("HWND"),
            _T("Captured Text"),
            _T("Font Name"),
            _T("Background Color"),
            _T("Foreground Color") };
        for( int i = 0; i < _countof( szColNames ); i++ ){
            m_lvTextSnasphot.InsertColumn( i, szColNames[i], 
		LVCF_TEXT|LVCFMT_LEFT, 120, i == 0 ? -1 : i + 1 );
        }
        return TRUE;
    }
 
    // If this line was reached - TextGRAB SDK initialization was not successful.
    EndDialog( 0 );
    return FALSE;

5. Text capture demo dialog member methods implementation

We need to unregister selection notification window before exit:

void CdemoDlg::OnClose()
{
    // Unregister selection notification window.
    m_spITextGRABSDK->RegisterSelectionNotify( NULL );
 
    CDialog::OnClose();
}

After checking “Enable capture” check box state it is necessary either enable or disable selection tool:

void CdemoDlg::OnBnClickedEnableCapture()
{
    // Enable selection based on checkbox state.
    CButton* pBtn = ( CButton* ) GetDlgItem( IDC_ENABLE_CAPTURE );   
    m_spITextGRABSDK->EnableSelection( pBtn->GetCheck() == BST_CHECKED ? VARIANT_TRUE : VARIANT_FALSE );
 
    // Enable "Select & Capture" button.
    ( ( CButton* ) GetDlgItem( IDC_SELECTANDCAPTURE ) )->EnableWindow(
            pBtn->GetCheck() == BST_CHECKED ? TRUE : FALSE );
}

Also it would be nice to have ability to start selection without hotkeys and only by clicking on some button on dialog:

void CdemoDlg::OnBnClickedSelAndCap()
{
    m_spITextGRABSDK->ForceStartSelection(VARIANT_TRUE);
}

It is necessary to show text snapshot list view only if corresponding capture method is used. In the other cases simple edit box is used:

void CdemoDlg::OnCbnSelchangeComboCapMethod()
{
    BOOL bShow = m_ComboCaptureMethod.GetCurSel() == 1; 
    m_lvTextSnasphot.ShowWindow( bShow ? SW_SHOW : SW_HIDE );
    m_Text.ShowWindow( !bShow ? SW_SHOW : SW_HIDE );
}

6. Selection handler and text capture functionality implementation

When user finished the selection – OnSelection function is called. At the beginning of function we get selection information, than check text capture method selected and after this – capture text using selected method:

LRESULT CdemoDlg::OnSelection(WPARAM, LPARAM)
{
	HWND hwndSelected = 0;  // Selected window handle.
	RECT rcSelected = {0};  // Selected rectangle coordinates.	
	CComBSTR bstrResult;  // Text capture results.
 
	// Get selection results.
	m_spITextGRABSDK->GetSelection( ( TGSDK::INT_PTR* ) &hwndSelected, &rcSelected );
 
	// Check selection mode and capture.
	if( m_ComboCaptureMethod.GetCurSel() == 0 )
	{
		// Capturing text in selected rectangle.
		m_spITextGRABSDK->CaptureFromSelection( ( TGSDK::INT_PTR ) hwndSelected,
			&rcSelected, &bstrResult );
	}
	else if( m_ComboCaptureMethod.GetCurSel() == 1 ) 
	{
		// Capturing text snapshot.
 
		// Perform capture.
		CComPtr< TGSDK::ITextSnapshot > spSnapshot;
		m_spITextGRABSDK->CaptureTextSnapshot( ( TGSDK::INT_PTR ) hwndSelected, &spSnapshot );
 
		// Get items count.
		LONG count = 0;
		spSnapshot->get_Count( &count );
 
		// Remove old data from list view.
		m_lvTextSnasphot.DeleteAllItems();
 
		// Enumerate all items in snapshot.
		for (int i = 0; i < count; i++ )
		{
			// Get next item.
			CComVariant varTxtItem;
			spSnapshot->get_Item( i + 1, &varTxtItem );
			CComQIPtr< TGSDK::ITextItem > spTxtItem(varTxtItem.punkVal);
 
			// Get item text.
			CComBSTR text;
			spTxtItem->get_text(&text);
 
			// Get item bounding rectangle.
			RECT rc;
			spTxtItem->get_Bounds(&rc);
 
			// Get item font.
			CComPtr< TGSDK::ITextFont > spTxtFont;
			spTxtItem->get_TextFont(&spTxtFont);			
			CComBSTR fontName;
			spTxtFont->get_FaceName(&fontName);
 
			// Get item window.
			TGSDK::INT_PTR window;
			spTxtItem->get_Window( &window );
 
			// Get colors.
			INT foreColor;
			INT backColor;
			spTxtItem->get_ForeColor(&foreColor);
			spTxtItem->get_BackColor(&backColor);
 
			// Format bounding rectangle for display and add it to list view.
			CString s;
			s.Format( _T( "%ld,%ld,%ld,%ld" ), rc.left, rc.top, rc.right, rc.bottom );
			m_lvTextSnasphot.InsertItem( i, s );
 
			// Format window handle for display and add it to list view.
			s.Format(_T("0x%xld"), window);
			m_lvTextSnasphot.SetItemText( i, 1, s );
 
			// Add the text and font name.
			m_lvTextSnasphot.SetItemText( i, 2, text );
			m_lvTextSnasphot.SetItemText( i, 3, fontName );
 
			// Add item colors.
			s.Format(_T("%ld"), backColor );
			m_lvTextSnasphot.SetItemText( i, 5, s );
			s.Format(_T("%ld"), foreColor );
			m_lvTextSnasphot.SetItemText( i, 4, s );
		}
	} 
	else if ( m_ComboCaptureMethod.GetCurSel() == 2 )
	{
		// Capture string.
		LONG index= -1;  // Receives character index in the string.
		m_spITextGRABSDK->CaptureString( 
				( INT_PTR ) hwndSelected, 
				rcSelected.left, 
				rcSelected.top,
				&bstrResult,
				&index );
	}
	else if( m_ComboCaptureMethod.GetCurSel() == 3 )
	{
		// Capture whole window text.
		m_spITextGRABSDK->CaptureFromHWND( ( INT_PTR ) hwndSelected, &bstrResult );
	}
 
	// Check if something was captured and set the results.
	if( bstrResult ){
		m_Text.SetWindowText(  bstrResult );
	} 
	return 0;
}

Conclusion

At this point you have working TextGRAB SDK demo. Now you can launch it and try how it works. You can use this demo as a base for your own C++ applications which need to have text capture functionality in it.
Download c++ text capture demo source code.
If you have any questions – please feel free to contact us.