Site configuration

The Site Configuration and Registration option includes several settings, impacting the overall system.

Localization handles your language, currency, default import, and internationalization settings. Further on in the Configuration Checklist page, you can change the address settings to your local requirements. Date formats can be set in Administer | Localization | Date Formats.

In the Country and State/Province sections within Localization, you choose the countries and their corresponding states/provinces that are available for addresses in your system. In most of the cases, you will select the same set of values for each field. If you have a site with multiple countries enabled, be sure to include the countries field in any forms (profiles) you create, and place it before the State/Province field. The State/Province field will generate a list of available values based on the user's selection of a country, which means that you want to encourage people to select the country value first. Failure to do this will result in a very long list of states/provinces for all the enabled countries, which will be cumbersome to work with.

CiviCRM has substantial multilanguage capabilities, including the ability to have multiple languages supported within the same site (allowing the user to select their preferred translation). You must install an appropriate language file if you want to use a language other than US English as your default. Refer to the discussion in the Installing CiviCRM section for details about the CiviCRM translation server.

Note that if you choose to make your site multilingual (where more than one language is available to users), it will require sufficient database user permissions, and will make modifications to your database that cannot be undone. If you are unsure if you want to use this capability, consider testing the functionality on one of the CiviCRM demo sites before implementing on your own site. You can also choose to keep the system single-language enabled initially, and then revisit at a later date if you choose to make it multilingual.

The Organization Address and Contact Info option is used to configure the name of your organization, the default FROM Email Address, mailing address, and other contact details. The settings here are important, as they will be used as the default FROM address for system-generated and mass e-mails sent from CiviCRM. Note that the e-mail address used generally needs to be a valid user on your hosting server in order for CiviCRM to send e-mails properly (this requirement will depend on your hosting environment configuration).

If you are planning to use CiviMail for broadcast e-mail capabilities from your site, be sure to complete the domain address fields so that it can be included in bulk e-mails. This is a good practice for any legitimate broadcast e-mailer, and is required by anti-spam legislation in some jurisdictions (for example, the American CAN-SPAM Act).

The Enable CiviCRM Components option is where you determine which of the major functionality areas in CiviCRM you plan to use.

CiviCRM is fairly modularized. By this, we mean you can turn on and off the main functionality sections (what CiviCRM calls components) in the system. This is particularly helpful for keeping a simplified, streamlined interface that only presents tools you are actually using. For example, if your organization doesn't host any events, you will not need the CiviEvent component and can disable it using this page. All of the menu entries for disabled components will disappear from the interface.

You can return and re-enable a component at any point in time. During this initial configuration, decide on the components that you know you are going to use and configure, and make sure that they are enabled.

Note that CiviContribute is less independent than the other components. CiviEvent, CiviMember, and CiviPledge all make use of the CiviContribute functionality. For example, if someone registers for an event with fees, the payment transaction is logged in CiviContribute, while the registration is logged in CiviEvent. If you plan to collect any type of payment on your site (events, donations, or membership fees), be sure to enable CiviContribute.

Lastly, Register your site will take you to the CiviCRM website where you can register your site. Registering does a few things that help the CiviCRM core team and project better understand who is using the system and how they are using it. In addition to basic details about your organization and whether you are evaluating or actively using the software, registration details what CMS you are using, how large your organization is, what sector your organization serves, what components you are using, and optionally, whether you are comfortable with CiviCRM using this information in marketing efforts. Please take a few minutes and register your site, as it will help the software better serve organizations like yours.

Viewing and editing contacts

The next configuration section, Viewing and Editing Contacts, handles settings related to contacts, including the contact record page, search settings, contact types, and address settings. We will walk through each of these areas.

The Display Preferences option controls what content areas are visible in different parts of the system. For example, if you are not planning to use tags as a categorization tool, you should remove it from your views. If you don't need to have immediate access to the changelog (which records a running list of when edits to a contact record were made, with the name of the user and timestamp), you can remove it from Advanced Search and contact record view.

The preferences are broken into four areas, with a few additional options throughout:

• Viewing Contacts: This option controls the tabs that appear across the top of the contact record.
• Editing Contacts: This option controls elements that will appear in the contact edit view (accessed from the contact summary tab). Note that some of these can be reordered by dragging and dropping the arrows icon.
• Contact Search: In this option, the advanced search groups fields by their type. You expand a group panel in order to access its respective fields. Use the options to disable panels you don't want to be displayed.
• Contact Dashboard: CiviCRM has a special page, which can be exposed to logged-in website users called the Contact Dashboard. The dashboard summarizes the user-related records, including their membership history, event registration history, donation history, and pledge history. In addition, users may see related contacts that they are permissioned to edit. Use the options here to configure the blocks that will be enabled.

As you initially configure this section, you may be uncertain about what options to select. Remember that you can always return at a later time to modify your settings. You will find whether you need, or don't need, certain elements listed here, as you progress through the site implementation and begin working in the system.

Additional configuration options on this page are included as follows:

• Viewing Smart Groups: Smart groups are saved searches; instead of simply collecting records into a box as with regular groups, this option saves the search parameters and reruns the search when the group contacts are retrieved. You can choose how to view these within the contact record with this setting. Generally, you will only want to list the contact's smart groups on demand, as the queries can be very resource demanding and should only be triggered when needed.
• Check for Similar Contacts: This option triggers a contact search as you fill out the new contact form, to alert you if there is an existing record that may match the contact you are adding.
• Notify Activity Assignees: This option determines if contacts assigned to an activity should receive an e-mail notification when the assignment takes place.
• Include ICal Invite to Activity Assignees: Choose this option to include an ICal invite when activity assignees are notified by e-mail. The ICal file can be used to easily add the activity to the person's calendaring system.
• WYSIWYG Editor: The editor toolbar will appear on any HTML text area in your site. CiviCRM is best integrated with the CKEditor toolbar, but you can choose from any in the list.
• Enable Popup Forms: This option will cause many of the forms in CiviCRM to open in a modal window rather than redirecting to a completely different URL. For example, when in a contact record, if you add a new membership, this option would open that form in a popup. Enabling this option can speed up the system and improve the experience for the end user.
• Individual Display Name Format: This option allows you to use tokens to configure how the individual's display name will be constructed.
• Individual Sort Name Format: This option allows you to use tokens to configure how the individual's sort name will be constructed. The sort name is what appears in search results, and is most often constructed using "last name, first name". If, for example, you want to include the middle name in this construction, you could modify the setting accordingly.

The Address Settings option provides similar functionality, allowing you to choose what is available for mailing label generation and the contact's address block.

Common changes to this page include removing {contact.supplemental_address_2} from the mailing label, adjusting the address viewing options (most organizations operate with two line addresses), changing the sequence of city and postal code for some countries, and hiding the Addt'l Address 2, Latitude, and Longitude values. Generally, you will rely on the geocoding function to populate the latitude and longitude values without the need for user involvement.

If you are interested in producing reports that are sorted by address elements, you should turn on Street Address Parsing. This turns on a functionality that attempts to automatically figure out the street number, street name, and unit number from the street addresses entered into the system. Currently, American, Canadian English, and Canadian French street addresses are supported. There are some extensions available that support other formats.

The last setting on this page handles address standardization. CiviCRM will integrate with the USPS web tools API to validate the address entered (this is for the United States only). Note that the tool only checks whether the address is valid; it does not determine whether the contact name is on record at that address. Before completing this section and enabling the tool, you must apply to the USPS and receive an ID and URL. This can be done at https://www.usps.com/business/web-tools-apis/welcome.htm.

The next configuration item in our checklist is Mapping and Geocoding. CiviCRM supports automated geocoding of addresses through Google and Yahoo!, and mapping through Google and OpenStreetMaps. If you are using Google for the mapping, you do not need to obtain a provider key, though you may choose to if you open an account to support higher volumes than Google provides for free.

When configured, CiviCRM will geocode the addresses as they are created or edited. When viewing a single contact record or search result list, you can view a map of the address or multiple addresses (if you are looking at search results), while taking the advantage of other functionality provided by the mapping service, such as plotting directions to the location.

• http://code.google.com/apis/maps/signup.html
• http://api.search.yahoo.com/webservices/register_application

Moving on in our Configuration Checklist page, the Search Preferences controls the fields that are searchable using the simple search tool, and how the search functions operate and display results. The options are as follows:

• Automatic Wildcard: This option enables the user to enter partial text and have search results return any matches in which that text appears, assuming wildcards before and after the search string. Larger databases may want to disable this option as it will slow the search performance and return a larger resultset.
• Include Email: This option searches both contact names and e-mail addresses for the search text.
• Include Nickname: This option searches contact names and nicknames for the search text. If you are a heavy user of nicknames, this is a very valuable feature. If not, you might consider disabling it as it slows down the search processing if you have a large number of contacts.
• Include Alphabetical Pager and Include Order By Clause: These options are useful features that facilitate navigation through search results. However, they can both impact performance on large databases. If you are experiencing poor response times while searching, consider turning one or both of these off.
• Default Contact Search Profile: After conducting a search, the result list table displays the contact name, address, phone, and e-mail. If you would like to set an alternative set of fields to be displayed by default, you must first build a profile (a collection of fields), and then select it from the Default Contact Search Profile option.
• Smart group cache timeout: This option controls how frequently the contact records in a smart group are cleared from the cache. A smart group is a saved search; caching improves performance when one is pulling the list of records from the search. By default, the cache is cleared every time a contact record is added or edited. This ensures that any changes to records are reflected in the dynamic search results. Recreating the cache, each time it is cleared, takes time, especially if you have a large number of contacts in smart groups. If you don't need up-to-the-second accuracy on smart groups, you can improve page load times on contact saves by increasing this value (5 minutes is recommended).
• Autocomplete Contact Search: At the top left of the horizontal navigation bar is a quick search field. As you type in the field, the system will display possible matches for the value typed. The possible match display will always include the contact name, but may also include e-mail, phone, and address fields. Use the Autocomplete Contact Search option to select the fields that should be included. Adding options can be helpful, as you can distinguish between contacts with the same name based on their e-mail, or any other value. Don't select too many options, however, as it will clutter the results drop-down menu and make it difficult to read the list.
• Contact Reference Options: Similar to the previous configuration setting, this option controls what fields are included in search results for contact reference type custom fields.
• Autocomplete Results: By default, any autocomplete fields (such as the navigation bar quick search) will only display 10 results. You can increase or decrease that count with this setting.
• InnoDB Full Text Search: One of the custom searches available with CiviCRM is a full text search. As the name suggests, this tool lets you enter a search term and search across multiple tables for the value. Let's say you are looking for all occurrences of a certain phone number, but realize it may not only be stored in the contact's phone number field, but also in the details field of several activities. The full text search will help you find all occurrences, regardless of where it was stored. By default, the full text search uses the standard MySQL queries to search through columns in various tables. While effective, it can be very slow, and may require significant server resources on larger datasets. This option will make use of the InnoDB full text search capability that was included in MySQL v5.6 to perform the search. This will drastically reduce load on your server, and also makes the search a bit fuzzier, as it seeks to locate all instances of the search text.

The next configuration page in our checklist, Misc (Undelete, PDFs, Limits, Logging, Captcha, etc.), is a catchall for various options relating to system operations.

The dashboard is the CiviCRM home page displayed when you first enter the system. You may configure the dashboard to display reports, or dashlets, providing a helpful summary of system data on a single page. Since the dashlet reports may pull from just about any data in the system, depending on how you've configured it, it may take some time for the page to load. System caching improves this by taking a static snapshot of the report data, and only reloading from live data on a periodic basis. You can control how frequently the cache is emptied with the Dashboard cache timeout option.

One really valuable feature in the system is the ability to send certain links with a checksum token. The checksum is a unique value generated for a specific contact. When included in certain URLs and sent from the activity e-mail or bulk e-mail tool, it allows the recipient to click the link and view the form as if they were logged in. In other words, it provides a way for people to interact with their contact record directly, as if they had logged into the system. This is invaluable for membership renewal campaigns, or for encouraging people to log in and update their contact details in the system using a profile. Because of the potential security risks associated with having a link that can interact directly with the individual's data, the checksum is given a finite lifespan, after which it will expire and prevent the user from interacting with their contact record directly. You can configure when it will expire using the Checksum Lifespan option.

The Contact Trash & Undelete option is useful for organizations that need to have stricter control over the deletion of contact records, including their contribution information. It can be disabled to simplify workflows and improve performance, but generally, you will want to keep it enabled. When enabled, a contact that has been deleted is first trashed; this removes it from the search results (by default) but retains it in the system where it can later be deleted permanently or restored.

By default, whenever a contact record is created or updated, CiviCRM records that a change was made and will list it on the contact record's logging tab. However, that basic functionality only records the fact that a change took place, who made the change, and the timestamp. It is also limited to actions such as the contact details, e-mail, and phone—it does not take into account many of the linked records (such as memberships, and cases) connected to the contact. Enabling the Logging feature implements enhanced logging functionality that will capture every change made in the system and provide detailed reports of exactly what changed. For example, if you update a person's phone number, you will be able to see a report that clearly shows you what the change was.

The enhanced logging functionality is achieved by creating a second set of tables that correspond to the main database tables. Every time a change is made in the system, a snapshot is stored in these logging tables. This way, the entire history of changes can be stored and reported on. One important thing to note is that by default, enabling logging will create those tables in your main CiviCRM database. This will bloat the size of the database considerably (as you are essentially duplicating the number of tables present). In the civicrm.settings.php file, you have the option of setting an alternate location for the logging tables. We strongly recommend you create another database specifically for use with logging, and update your settings file accordingly before you enable logging.

The Attach PDF Copy to Receipts option, as the name suggests, will generate and attach a PDF file to e-mail receipts generated by the system.

PDF generation is available throughout the system in various places—you can: generate a PDF letter (a merge letter) from a contact record or search results; generate a PDF file from reports; and create PDF invoices and receipts. While these tools are all very useful, they suffer from one inherent problem: the PDF generation process is very resource intensive and consequently is very susceptible to browser timeouts. One way to reduce the likelihood of resource problems is to use the alternate wkhtmltopdf library. Use the Path to wkhtmltopdf executable option to designate the location of the wkhtmltopdf executable. Note that wkhtmltopdf must first be installed on your server, as it is run through the command line. Links and details are available from this configuration page.

The Automatically Check for Updates option helps you, as an implementer, to keep a track of the latest release of CiviCRM. If your installed version is older than the current stable release, CiviCRM will place a notification on the administration pages to alert you. If this option is turned on, basic statistics will also be sent back to CiviCRM, so the core development team has a basic sense of what version people are using, what environment it's being hosted on, and the size of databases. These statistics do not contain any data from your site—they only include the aggregate statistics of your data and system environment information. Wherever possible, leave this turned on, as it is very helpful for the core team to understand how the community is using the software, and often, it will inform the direction of future development. There are several other options related to the update check:

• Ignore Updates Prior to: Using this option, set a date in this field to ignore updates that were released prior to the date. This is a simple way to temporarily suppress the update notification if you do not wish to be reminded of it.
• Security Update Alerts: Using this option, designate where you would like to see update notifications for security releases.
• New Version Alerts: Using this option, designate where you would like to see update notifications for non-security related releases.

The Display "Empowered by CiviCRM" option will place a logo and tagline in the footer of any public facing pages of your site. This is a simple way to let people know that you are using CiviCRM and helps draw attention to the project.

While generating e-mails or creating activities, users are allowed to attach files. Set the Maximum Attachments option here to prevent too many from being attached at a time. You can also set Maximum File Size (in mb) to limit how large a file can be attached. Keep in mind that your PHP settings will also place limits on the maximum file size, and should be taken into consideration, or adjusted, when configuring this option.

Lastly, you may enable anti-spam tools for your online forms using reCAPTCHA. The reCAPTCHA option helps ensure the form is completed by a human and not a computer. It presents words, or an alphanumeric string in a graphical form, usually with some kind of warping or other visual obstruction that is hard for computers to decipher. If you plan to have publicly available forms on your site allowing users to enter data, you should consider implementing this.

First, sign up at http://recaptcha.net/ to obtain a public and private key. Then, return to this page to complete the form. Once configured, you enable reCAPTCHA on a form-by-form basis in the Advanced Settings panel of your profile's Settings page. We will discuss this further when we cover profile forms and the process of exposing data to your website.

The last configuration page in this section is Contact Types. CiviCRM has three primary contact types—Organization, Individual, and Household. Contact types are used to distinguish the types of records you maintain. Each type will have some unique data fields, in addition to fields common to all. For example, the first name and last name fields are appropriate for individual records, but not for organization records, such as a business. In contrast, address fields are common to all.

Beyond the three primary contact types, you may create additional subtypes to categorize your records. You may also rename the existing primary types using this configuration tool. There are a few important things to remember when thinking through contact subtypes:

• There are several benefits in using subtypes to organize your records, but chief among them is the ability to create custom fields assigned to one or more subtypes. Consider a school that is using CiviCRM to track both parents and teachers. These are both individual contact types, but the data collected for each would be very different. The school may want to know the skill sets for parents who wish to volunteer, or track if either parent is an alumnus of the school; teacher records, on the other hand, would track information about their educational experience, history of classes taught, and employment-relevant human resource data. Creating subtypes allows you to segment the data you are collecting.
• Subtypes extend one of the three primary contact types. When creating your subtypes, first think through which of the three main types the subtype should be based on.
• A contact record may be assigned multiple contact subtypes. You can construct them in such a way that you anticipate scenarios where a contact is potentially assigned multiple types.

For a more complete review of contact subtypes and additional guidance on how to think through your system structure, refer to Chapter 5, Collecting, Organizing, and Importing Data.

Sending e-mails

Having addressed core configuration and contact-related settings, we now turn our attention to e-mails generated from the system. CiviCRM may be configured to automatically generate e-mails for end user actions, such as registering for an event or making a donation. Additionally, administrators can send e-mails to one or more contacts using activities or CiviMail, the mass e-mail distribution tool. Though you are not required to use e-mail sending capabilities in order to use CiviCRM, you will miss out on significant functionality if you choose not to.

First, set up your Outbound Email configuration at Administer | System Settings | Outbound Email (SMTP/Sendmail), and then from one of the options present:

• mail(): This option makes use of the PHP mail function. It is generally sufficient for most transactional e-mail uses.
• SMTP: This option may provide more complete bounce headers for return mail processing, and provide the flexibility to optionally use external SMTP sending services. This may be something you want to consider if you are sending very large bulk e-mails, as it will reduce the responsibility you bear for maintaining good mailing reputation, as well as your server requirements. The mail() option is generally sufficient. Your hosting provider may also have a preference, or may only support certain options.
• Sendmail: This option is an alternative sending mechanism available on most *nix hosting environments.
• Disable Outbound Email: As the name suggests, this option will completely disable the sending mechanism. This option is most often used in a development environment where you want to ensure e-mails are not unintentionally delivered from the system.
• Redirect to Database: This option will prevent e-mails from being sent, but will capture the record of their intent to be sent in the civicrm_mailing_spool table. When this option is enabled, bulk e-mails will be archived when they are scheduled.

Regardless of the option you choose and configure, be sure to use the built-in testing function and confirm receipt of the e-mail. This quickly and easily ensures that the outbound e-mail is working.

Once the outbound e-mail starts working, configure the settings at Administer | Communications | FROM Email Addresses. When sending e-mails from CiviCRM, the logged in user's e-mail will be the default FROM address. However, you may configure the additional addresses available to the system and select them while sending an e-mail. These e-mails will also be available as the sending address for bulk e-mails.

For example, your organization might have service-based generic e-mails, such as events@my.org, questions@my.org, and membership@my.org. Depending on your hosting setup, the FROM e-mail addresses used may need to exist in your mail server as mailboxes or forwards.

Handling return e-mail traffic

The CiviMail component provides important functionality for processing inbound e-mails. These e-mails are received and processed by the system for three purposes:

• The first purpose is the bounce handling of invalid e-mails and temporarily undeliverable e-mails. Without this functionality, you increase the risk that ISPs will, at some point, label your server a spammer. Prevention is the best medicine, as it can be difficult to track whether you have been blacklisted, or to respond quickly and effectively if you have been. This isn't merely a reputation problem: when your e-mail is not being delivered, it can significantly impact donations, membership signups, renewals, and event attendance.
• Secondly, return mail processing provides support for e-mail-based unsubscribes, opt-outs, and resubscribes. When bulk e-mail is delivered to recipients, your e-mail should include options to unsubscribe from the mailing list, opt-out of all e-mails, and potentially a link to resubscribe (if they have unsubscribed). Depending on how you set that up, you can support those actions through e-mail, in which case they need to be processed by the system.
• Finally, CiviCRM supports the ability to receive e-mails and log them as Activity records associated with contacts in the From, To, and CC lines. As you generate e-mails from your web-based or desktop e-mail client, you can automatically attach a copy of the e-mail to existing or new contact records using this functionality.

Unfortunately, configuring the return e-mail channel can be tricky, and may be one of the more technically challenging aspects of installing and configuring CiviCRM. It is much easier now than it was a few years ago, but is still an area where you may want to obtain professional support to ensure proper functioning.

Alternatively, CiviSMTP (http://www.civismtp.com) is a paid service that takes care of most of the technical challenges involved in setting up and running CiviMail, and may help you overcome obstacles you run into setting it up on your local server. Note that it is not an official service offered by the CiviCRM project, but is owned and operated by a third-party provider.

Before walking through the configuration of the return channel processor, let's review the broader subject of maintaining a good mailing reputation.

Maintaining a good e-mail server reputation

Any broadcast e-mails sent by your system will be monitored by various spam detection systems operating on the Internet. While you do not have direct control over those systems, you do have the ability and responsibility to construct and manage your broadcast e-mail systems in such a way so as to minimize the likelihood that you will be flagged as a spammer. This process of reputation management involves domain record configuration, including sender details in your e-mails, providing unsubscribe and opt-out tools, and processing bounced e-mails so that you are not repeatedly mailing to bad e-mail addresses, and so on. Each of these things will help contribute toward lower spam scores and ensure you are able to continue reaching your constituents effectively through broadcast e-mails. In this section, we will take a look at Domain Name System (DNS) related considerations.

The first step is to set up an SPF record in the DNS for your domain, and ensure that it is correctly configured whenever you move servers. The Sender Policy Framework (SPF) identifies, to the world, hosts that are specifically allowed to send e-mail from your domain, those not allowed, and others about which nothing can be said. Your CiviCRM server and domain's outbound mail server (if different), should both be identified in the SPF as allowed to send (pass). The more specific and limited your SPF record is, the more effective it will be in contributing toward reducing CiviMail deliverability issues. Unfortunately, users who travel with laptops through various IPs while sending e-mails may make it difficult to lock down the sender policy framework tightly.

A second useful approach to improving your server's reputation is to set up a reverse DNS. In the Sending e-mails section, you should have configured and successfully tested your outbound e-mail settings. In order to determine whether everything is really working correctly, use CiviCRM to send an e-mail to an account you can access that is not on your server, for example a Gmail or a Hotmail account. Then, examine the e-mail header for a couple of things. In Gmail and Google Apps, you can do this by performing the following steps:

1. Opening the message.
2. Clicking on the other actions dropdown beside Reply or Reply to all.
3. Selecting Show original.

Look for a Received-SPF header record. It should indicate either neutral or pass. You may also find other records such as Authentication-Results: spf=neutral.

If you are having trouble with SPF, the issue can sometimes be traced to the identity that is provided by, or about, your server when it sends e-mails. Check through the Received: From header records until you get to your originating server, and perhaps beyond it, to your domain on a localhost. Raise any issues or questions you have with the tech support at your ISP.

Reverse DNS is the process of translating IP addresses to hostnames, the opposite of the forward DNS process, which takes the commonly-used domain name and translates it to an IP address. Setting up a reverse DNS record (if not done so already) can help improve your e-mail reputation as the sending IP is validated as a legitimate sender for your domain. If your server has multiple domains sharing a single IP, it may be worthwhile to spend the small amount of money necessary to purchase a dedicated IP (useful for SSL certificates for CiviContribute, as well) so that you can set up the reverse DNS for your domain. A shared IP environment may result in your domain being scored poorly if it resides in the same bank of IPs as spammers, or other accounts that are not as careful with their e-mail reputation management.

The third approach to improving your server's reputation is to monitor the DNS blacklists and to request organizations that maintain these lists to remove your server from the list, if it is present. You may find yourself listed for a variety of reasons, legitimate or not, including: you haven't been processing bounces properly; recipients of your e-mail may make mistakes in their complaints; or even other sites affecting your reputation. It is important to respond to blacklisting promptly, as it typically propagates from one service to another.

Configuring the e-mail processor

To process the inbound e-mails, you must have an e-mail account to receive the inbound e-mails and configure a scheduled job to periodically process the e-mails received (addressed later in this chapter).

Although not required, you will typically want the e-mail account used for bounce processing to be dedicated for this purpose. You also would not typically want it to be a publicized account that constituents would send e-mails to. It should be created and configured solely for the purpose of bounce mail handling.

CiviMail uses Variable Envelop Return Path (VERP) processing as the method for tracking bounces. The method adds a unique sender address to every outbound message. When a bounce is returned, this unique sender address is used to determine the message, e-mail address, and contact record that generated the bounce. Bounces are classified by type (whenever possible) and processed based on the rules configured in the system. Different bounce types will trigger a temporary or permanent hold placed on the e-mail, or record the delivery attempt for future processing, should there be additional bounces on the address.

The preferred setup for VERP is called subaddressing, which uses a local part format. If your invisible bounce processing e-mail address is civimail@example.org, the VERP address appends a unique value after the separator but before the domain, such as civimail+12.234ka.241123@example.org. While this is the preferred method, some ISPs deliberately prevent subaddressing on their servers because of its prevalent use in broadcast mailing.

In order to test whether you can use this approach, try sending an e-mail to the account you have set up, appending +test or –test to the account before the @ symbol (myemail+test@mydomain.org). If you receive the message successfully, you've confirmed support for local part handling and are ready to set up the account for inbound message processing.

If subaddressing/local part handling is not supported, or you prefer not to use that method, you may still be able to process bounced e-mails using a catch-all e-mail account. A catch-all account is configured to receive all the e-mails not addressed to the other mail accounts set up on your domain—it catches any e-mails not caught by the other mail accounts. In general, catch-all accounts are less desirable as they will typically catch quite a few spam e-mails, which may be processed along with legitimate bounces. To determine whether your hosting environment supports a catch-all account, log in to the system that administers your e-mail accounts and look for the option to designate an e-mail as the catch-all, or contact the hosting support to determine the system capabilities. If your system supports catch-all addresses but not subaddressing, you'll have a slightly higher load on your server, but the system should function fine.

Tip

In some cases, you may find a situation where the VERP-based processing does not work for your environment and needs. For example, if you are using an external service for SMTP delivery, they may require that all bounce processing be handled by their service—which means you will not receive bounce e-mails in your bounce account. In such cases, you may have two options (depending on their setup): you can integrate with their service via the API, retrieve/receive bounces, and then process those in CiviCRM. That will require a fair bit of custom programming, but may be worth it, depending on your usage. Another option, some SMTP processors provide, is to forward bounces to an inbox after they have been handled by the service provider. While this will help get the bounced e-mails to your inbox, it does not solve the problem. As those e-mails are being forwarded, the delivery address does not include the unique values that help CiviCRM identify the e-mail and mailing they originated from. One way to work around this is to include a Message-ID header in the mailing. Visit Administer | CiviMail | CiviMail Component Settings and enable this option. This will embed those e-mail/contact/mailing specific details in the e-mail header, which will be passed along when forwarded and made available for processing. Be sure to test this approach thoroughly to ensure it works as expected in your system.

Many CiviCRM installations run on virtual private servers. In most cases, the administrators of VPSs have the authority to configure their e-mail systems to their own needs and preferences. If you don't have the technical skills yourself, it's worth putting in a support request to have things set up the way you want.

The final way to support VERP, if your e-mail system can't support either subaddressing or catch-all addressing, is to use a third-party account such as Gmail. The processing time and resource load to support processing remote e-mail stores is higher, and there may be a slight hit to your reputation if the sender domain is different from the FROM e-mail domain; but this is still a viable and effective solution. Note that depending on the system used, you may need to configure filters so that bounced e-mails are not flagged and filed as spam. For example, Gmail accounts should have a filter created that redirects anything flagged as spam back into the inbox so that it can be processed by CiviCRM.

Tip

Gmail (and other providers) may also place restrictions on your ability to remote connect to the account from the server. Be sure to log into the account and address any notifications you receive, and also monitor the processing logs to make sure they are returning a success response.

Configure CiviMail with the bounce processing account you have set up as follows:

1. Navigate to Administer | CiviMail | Mail Accounts.
2. Click on the Edit link on the default account row.
3. Alter the Name option as desired.
4. Enter the Server, Username, Password, and Email Domain fields for the account, along with any other configuration options appropriate for your setup.
5. If the account supports subaddressing, enter the e-mail address name+ in the Localpart field (include the plus sign, such as civimail+).
6. If possible, configure the system to use IMAP in the Protocol field (preferred over POP3 or other options). Use your e-mail client to test whether the account is working properly before expecting CiviMail to talk to it.
7. Check the e-mail client to test whether the account is working properly before expecting CiviMail to talk to it.
8. If possible, set up SSL to improve the security. A locally issued certificate should be fine.
9. Select the Bounce Processing in the Used For? field.

Once configured, you can manually trigger the scheduled job by visiting Administer | System Settings | Scheduled Jobs, finding the Fetch Bounces job, and under the more popup select Execute Now. You can then select View Job Log to review the processing response. After you've confirmed it is working, you can enable the job and configure its frequency. It's typically sufficient to run it on an hourly basis. Remember that scheduled jobs must be triggered by a Cron job in order to run automatically (discussed later in this chapter).

Note that the configuration guidance outlined in the preceding steps covers the standard setup options for bounce processing. Other configuration options are available, such as return-path headers and alternate protocol options, which may be used if your server setup is nonstandard. For more details on how these options may be used, review the online documentation or search for guidance in StackExchange (http://civicrm.stackexchange.com/).

Payment processors

It's likely that one of the reasons you've decided to implement CiviCRM is to benefit from the ability to process payments through your website, and have it immediately recorded in your contact management system. Whether you are collecting donations, campaign contributions, sponsorship commitments, pledges, membership fees, or event registration fees, all of those transactions will flow through CiviContribute.

However, before you can collect and process payments through CiviCRM, you must configure a payment processor. Your payment processor or the payment gateway is a service you purchase to allow credit card processing either directly through your site or through redirection to the processor's site. The processor receives the credit card, authorizes the payment, and transfers the funds to your bank (either manually or automatically, depending on the service).

Note

To be more precise, you will need to set up a payment processor if you plan to receive and process transactions directly through your site. If you only receive offline payments (checks, cash) or process credit cards through other, nonintegrated means, you can still make use of CiviCRM contribution management tools.

Choosing what service(s) you will use can be challenging, as there are many options available. You will want to make sure that whatever service you choose has a corresponding CiviCRM payment processor support available; otherwise, you may need to develop a new extension to integrate with your selected service.

Currently, the following integrations were available in the core software:

• Authorize.net
• DPS Payment Express
• Elavon Payment Processor
• eWAY (Single Currency)
• FirstData (Linkpoint)
• Google Checkout
• Moneris
• PayflowPro
• PayJunction
• PayPal—Express
• PayPal—Website Payments Pro
• PayPal—Website Payments Standard
• Realex Payment

In recent years, there has been a shift to reduce what processors are supported through core and to begin to support them (and include community-sponsored development) through extensions. There are quite a few payment processor extensions available, including a few (such as iATS and Stripe) that are quickly becoming favorite options among many developers. If you do not see your extension in the preceding list, visit https://civicrm.org/extensions and view the options available there.

Not all payment processors are created equal. Some are only available in limited regions or have limitations on what currencies can be used. The structure and model for what fees are charged will also differ significantly. If you are planning to use recurring contributions, be sure to investigate whether the processor will allow such functionality and ensure that it has been built into the CiviCRM plugin.

Note

For more details about configuring some of the payment processors and the functionality they provide, visit http://wiki.civicrm.org/confluence/display/CRMDOC/Payment+Processors. Note that not all processor plugins are officially supported by the CiviCRM project, and not all are detailed in the preceding link. Some have been contributed by the community and rely on community support for ongoing updates.

Integrated versus redirection processors

There are the two basic types of payment processors, namely, those that allow you to collect and process credit cards directly from your site (integrated), and those that will direct you to the processor's site to handle the transaction (redirection). There are pros and cons to both:

• Those that allow credit card collection directly from your site provide a more attractive, branded solution. The user never leaves your site and the transaction process flows seamlessly from the collection form, to a confirmation page, and a final thank you/receipt page. However, the fees charged by the processing service are typically higher and you must have a dedicated IP with SSL installed in order to sufficiently secure the transaction.
• Those that direct you to the processing site (such as PayPal Standard and Google Checkout) do not require SSL (as the security is handled by the processing site), and can typically be implemented very quickly and may charge lower total fees. However, the redirection to the processing site means that visitors must leave your website to complete the transaction. This may discourage or confuse people from finalizing the payment.

Note

CiviCRM will not work with a shared SSL (commonly offered on many shared hosting environments). You must have the SSL installed on your domain in order for CiviCRM to work correctly. If you implement SSL on your site, be sure to visit Administer | System Settings | Resource URLs and enable the Force Secure URLs (SSL) option. This will redirect the contribution pages and the CiviCRM administration pages to SSL. You will typically also want to verify SSL Certs on that page.

If you are new to online payments, and are unsure how much your constituents will respond to the new functionality, consider starting with the off-site solution where you are directed to the processor's site to complete the transaction. At some point, you may reach a threshold where the additional cost for handling the transaction directly through your site is justified.

Tip

The SSL certificates are domain-form-specific. A certificate purchased for www.yoursite.com will not validate for yoursite.com, as the root form of your domain and the www form are not considered the same. In addition, CiviCRM references some resources through the site URL and may not function consistently if the root and www form are used interchangeably. The easiest way to address this is by adding the Apache directives to an .htaccess file in your public website directory. For example, if you wish to redirect from the root form to the www form, add the following code (with your domain name):

RewriteEngine On
# Redirect root domain to www. form
RewriteCond %{HTTP_HOST} ^yoursite\.com$ [NC]
RewriteRule ^(.*)$ http://www.yoursite.com/$1 [R=301,L]

Then, make sure all references to resources in your CiviCRM configuration use the www form of the domain. In addition to ensuring the proper function of CiviCRM and your SSL certificate, this will help create consistent domain branding of your website.

Configuring the payment processor

Once you've decided what payment processor to use and have gone through the necessary application process with the company you are using, you may set up the account in CiviCRM. From the Configuration Checklist page, select Payment Processors, or visit Administer | System Settings | Payment Processors.

The fields you must complete will depend on which processor you use. In almost all cases, CiviCRM will let you configure the credentials for your live processor and a sandbox (test) account. The sandbox credentials will be used when you use the test drive option in contribution pages. See your payment processor for details about creating a sandbox account.

Tip

If you do not yet have a payment processor account but would like to begin setting up and testing contribution pages or other payment handling forms, use the Dummy Payment Processor option. You may then configure your forms and enable contribution handling (including events and membership forms). No transaction is triggered when a dummy processor is used—it should only be created for temporary testing purposes.

CiviCRM supports the configuration of multiple processors. At the time you create a contribution page or event registration page, you will be asked to select your desired processor to be used by this form.

After configuring your payment processor, be sure to test it by generating a live contribution record. Don't rely on the sandbox/testing tools alone; always run a live transaction with a real credit card to test the system before exposing live donation/pledge/event/membership forms that will process payments to the public.

The System Workflow Templates page

After completing the setup of your payment processor, return to the Configuration Checklist page and visit the System Workflow Templates page. As people visit the CiviCRM forms on your site, and you, as an administrator, work with contact records, there are various e-mails generated by the system and sent to your contacts. Each of these e-mails can be customized through this page.

If you edit any of these e-mails and save the changes, you will have the option of viewing or reverting to the original version.

The templates contain tokens and variables that are placeholders for contact and record-specific field values. For example, if you want the e-mail to be addressed to "Dear First Name," you would insert the First Name token. When editing the template text, you may use the Insert Tokens button to insert basic contact fields.

Take care when editing templates. Most of these contain a number of conditional clauses (the if…else statements) in order to ensure that the resulting e-mail is customized to the specific transaction. After making edits, make sure you test it with various options to confirm that it is functioning properly.

Note

CiviCRM uses the Smarty template engine (http://www.smarty.net) to render pages, including workflow e-mails. You may insert basic logic and make use of other Smarty functions when creating your e-mail template.

Organizing, customizing, and components

At this point, you've completed the most essential steps in your initial site configuration. The Configuration Checklist page includes three additional sections that we will briefly review here, and cover in more detail later, while walking through the full implementation of their respective functions. These additional sections are Organize your contacts, Customize Data, Forms and Screens, and Components.

Organize your contacts

As you begin importing and entering contacts you will begin looking for ways to organize and categorize them. There are many ways to do this in CiviCRM, but the two primary methods are with tags and groups. Tags can be constructed in two ways: a hierarchal categorization tree you create in order to classify contacts, or a fieldset where you enter free-text tags (as you type you can search/select existing tags or create new tags). Groups are the collections of records that may be dynamically created (Smart Groups, the equivalent of a saved search) or manually collected (for example, an opt-in mailing list). We discuss configuration and implementation of these in Chapter 5, Collecting, Organizing, and Importing Data.

Customize data, forms and screens

CiviCRM's core data fields cover all the basic contact and communication fields you would expect. Inevitably, you will have the additional pieces of data specific to your organization that you want to record in the database. The Custom Data tool of CiviCRM allows you to attach additional fields to almost any existing type of record. For example, you can add fields to a contact record, contribution record, event registration, relationship type, and so on. We will look at creating custom fields in Chapter 5, Collecting, Organizing, and Importing Data.

In many, but not in every case, the data fields you've created will be included in online forms and exposed to your site visitors. We use Profiles to select core and custom fields for inclusion in specific forms. We will begin to look at constructing and using profiles in Chapter 5, Collecting, Organizing, and Importing Data.

Components

As noted earlier in this chapter, CiviCRM components represent the major functional areas of the software. Your first configuration step was to enable/disable the components you plan to use. This last block in the Configuration Checklist page lets you access settings and options specific to each component. As we explore each of these functionality areas throughout this book, we will examine the configuration options specific to each.

Option lists

Although we completed a review of the Configuration Checklist page, there are several additional settings we want to explore. You may choose to configure these areas now, or simply make note of them, and return to modify them later in your implementation process, as the need arises.

As you work with CiviCRM, you'll quickly find that there are numerous option lists providing selection choices for fields. These lists include location types, phone types, individual prefixes and suffixes, activity types, and many more. A full list of configurable options is found by visiting Administration Console and selecting Option Groups from the System Settings section, and a partial list of the most frequently modified options is available at Administer | Customize Data and Screens | Dropdown Options.

The available settings for each of the options will be specific to the option type. In most cases, it will be self-explanatory or, otherwise, documented with help text. If the option contains both a name and a label field, the name field is used for the unique option value, and the label field is what will display in the option list.

Tip

CiviCRM has begun to provide access to option lists within the areas where they are used (provided you have sufficient access). Look for the wrench icon, which indicates you can modify the list without navigating to the dedicated administration tool.

Synchronizing users with contacts

If your Joomla!, Drupal, or WordPress website has been in operation for some time, you will most likely have an existing list of website user accounts. These user accounts will not immediately have corresponding CiviCRM contact records unless the user has logged into the site and visited a CiviCRM page. As part of your initial system configuration, it is best to synchronize those CMS users with CiviCRM. Access this option by visiting Administer | Users and Permissions | Synchronize Users to Contacts.

Triggering this tool will create a linked contact record for each CMS user in your website. It should only be necessary to initiate this synchronization once. Thereafter, new CMS users will be linked to CiviCRM as they are created or visit a CiviCRM resource.

Note

There are various ways this will occur, depending on what CMS you are using, and based on what account creation tools you have implemented. We will review some of the available options later in this book.

Let's take a moment and understand exactly how CiviCRM contacts relate to the CMS users.

Your website (CMS) users are individuals with permission to log into the site using a username and password. CiviCRM contacts are records residing in your constituent database. CiviCRM creates a link between the CMS user and its corresponding CiviCRM contact record. When the user logs in to your site, you can then expose data fields specific to that contact. For example, you could enable your logged in contacts to edit their address and phone details.

While every CMS user should have a corresponding CiviCRM contact record, the reverse is generally not true. You will typically maintain contact records for individuals (and certainly organizations and households) that do not have a corresponding user account.

As you configure and plan your site, begin thinking through exactly who should have access to the site, what data fields you might expose to your website users for editing and updating, and what workflow you prefer for creating and granting permissions to new users.

Access control

One of the most important things you need to consider when configuring your system is to decide who will be allowed to access your contact data, and what tools they are permitted to use. This is handled with CiviCRM, in conjunction with your CMS, through access control settings.

All three integrated CMS's handle permissioning in a similar way—though there are a few differences between them:

• Users are assigned one or more roles/groups:
  • Drupal and WordPress use the term roles; Joomla! uses the term access control group
  • Drupal and Joomla! support assigning multiple roles to a user; WordPress only allows you to assign one
• Roles/groups are assigned permissions:
  • Drupal and WordPress assume the permission is denied unless it is explicitly granted. In Drupal, assigning multiple roles will result in the user being granted all permissions assigned to those roles.
  • Joomla! supports an access control group hierarchy, where groups further down the tree inherit from those higher up the tree. Permissions are assumed deny, and can be explicitly allowed or denied.

WordPress has several roles defined by default. You are not able to add additional roles unless you install additional tools to provide that functionality.

Access is provided to users based on their role/group that can be added or removed as a person takes on, or gives up responsibilities for accessing and administering functionality and content in CiviCRM.

Each CMS includes the concept of an anonymous or public user—someone who has not logged into the site (including those who do not have user accounts), and authenticated or registered user (anyone who is currently signed in). Additional roles are typically created for system administrators and for the groups of people who need to be allowed to do some things that others are not permitted to do, for example, moderating comments or other user-generated content.

We don't recommend creating an overly complex set of roles for CiviCRM. Ideally, you will set up permissions so that people are always allowed to do what they need to do, and aren't allowed to do things that are dangerous to the system, its data, or the organization's reputation for preserving the privacy of the personal data it holds. Often, you can get by with a single role called CiviCRM administrator, and perhaps with another one allowing users to view and edit financial information.

CiviCRM defines over 80 permissions used throughout the system. Many are used by specific components within CiviCRM, and so may or may not be applicable, depending on which components you are using. We will not list or detail the permissions here—most are fairly self-explanatory, though you will want to test your roles carefully to confirm they restrict or permit the desired functionality. You can access the permission administration by visiting Administer | Users and Permissions | Permissions (Access Control), and selecting WordPress/Drupal/Joomla Access Control.

Broadly speaking, permissions fall into several functional grouping:

• Accessing the components of CiviCRM (for example, accessing CiviMember)
• Performing certain operations on the certain types of objects (for example, adding contacts)
• Accessing administrative interfaces (for example, administering dedupe rules)
• Accessing non-administrative functions (for example, making online contributions and registering for events)

Advanced access control

In addition to the permissions that CiviCRM exposes to the CMS access control permissioning system we saw in the previous section, CiviCRM has its own access control capabilities that provide finer-grained access control. These tools provide powerful ways to further segment and control who is able to view and work with your contacts and records. But that power comes with a price: using these tools will complicate your system and you should anticipate the need to be monitoring and adjusting these rules to meet your users' needs; and secondly, advanced permission does have an impact on performance. You are basically adding a potentially complex layer of access control to every function in the system—when a contact is viewed, certain data is accessed, and so on—which will have an inevitable impact on performance.

CiviCRM advanced access control is broken down into three components:

• Role
• Operations (View, Edit, Create, Delete, Search, and All)
• Data

This provides a very flexible structure for controlling access. Let's walk through how to use these tools. Visit Administer | Users and Permissions | Permissions (Access Control) and review the three options:

• Manage Roles: As with the CMS-based access control tools, we begin by defining roles that will represent the types of access to be granted.
• Assign Users to CiviCRM ACL Roles: With this option, we decide the contacts that will be granted the defined roles. This is done through CiviCRM groups—specifically, those that have been flagged as an Access Control type of group. Any contacts added to the selected group will be granted the access permissions attached to the linked role.
• Manage ACLs: With this option, we define the access rules for different roles. Multiple ACL rules can be defined and associated with the same role.

ACL rules are defined by an operation, a type of data, and the data selections (based on the type chosen). Operations define what the person can do with the selected data, and are fairly self-explanatory:

• View
• Edit
• Create
• Delete
• Search
• All

Four options are given for Type of Data:

• A group of contacts: After selecting this option, choose the group for the the operation that should be applied. Leave the selection set to - select - to create an ACL for Role and Operation that applies to all contacts.
• A profile: This option is useful to lock down the use of certain profiles to certain purposes. When this option is selected, you can select the profile or leave the value at - select - to create an ACL that applies to all profiles. Make sure that all the users that are expected to complete online contributions, event registrations, or other similar functions such as newsletter subscription signups have the Create operation permission for all profiles on those forms.
• A set of custom data fields: When selected, you can select the Custom Data group of fields or have the permission to apply to all the custom fields. Note that there is a known limitation with these settings: the View operation on custom data fields also permits editing, and the Edit operation also permits viewing.
• Events: This option is useful to restrict users from viewing the event information page, or from registering (using the Edit operation) for events.

ACL permissioning can be tricky, as you must make sure that the roles assigned to users don't conflict with each other. If you are using CiviCRM's internal granular permissioning, you must also be careful that you don't conflict with the CMS/CiviCRM permissions assigned to the user via the role/access control group. For example, a user with an access all custom data permission would override a specific custom data set permission defined in CiviCRM.

As a general rule, take the time to do sufficient testing for any roles you've created, be it in the CMS or CiviCRM—especially, if you have sensitive information to which you must be sure to restrict access.

In the case of third-party extensions, if a user is assigned three roles, the combined permissions from all three roles will determine what the user can do in the system. As permissioning explicitly allows the specified access, if any of the three roles have been granted permission, it will impact the users' access.

There are several extensions available for each of the three CMSes that have an impact on ACL handling. We will briefly introduce and review them here as they are applicable to the current discussion.

Drupal – the CiviGroup Roles Sync module

This module allows all the users in a particular Drupal role to be placed in a CiviCRM group, and vice versa. This allows CiviCRM to put contacts into groups and have that result in additional access privileges in Drupal. For example, signups on the CiviCRM profile forms can be automatically added to groups, which could then be used to enable access to parts of your website. Smart groups can also be created for all donors, or other criteria, such as geography, to provide differential access to parts of your site.

In order to enable CiviGroup Roles Sync, follow these steps:

1. In Drupal, visit the Module administration.
2. Click on it to enable the CiviGroup Roles Sync module in the CiviCRM modules fieldset.
3. Click on the Save Configuration button.

In order to configure CiviGroup Roles Sync, follow these steps:

1. Create one or more non-default Drupal roles.
2. Create one or more CiviCRM roles at Administer | Users and Permissions | Permissions (Access Control) | Manage Roles.
3. Create a group of users to assign to CiviCRM roles, if necessary, under Contacts | Manage Groups. Note that Access Control must be enabled as Group Type in the Settings for the group.
4. Assign users in an access control group to each CiviCRM role at Administer | Users and Permissions | Permissions (Access Control) | Assign Users to CiviCRM ACL Roles.
5. In Drupal, navigate to Administer | Site configuration | CiviGroup Roles Sync.
6. Click on Add Association Rule.
7. Select a CiviCRM group and a Drupal role from the selection lists.
8. Click on the Add association Rule.

Drupal – the CiviMember Roles Sync module

This module provides two-way synchronization between specific membership types with specific membership status and Drupal roles. For example, this allows you to provide all the current members with access to a site, and have that automatically removed if they don't pay their membership renewal fee.

In order to enable the CiviMember Roles Sync module, follow these steps:

1. In Drupal, visit the Module administration.
2. Click on it to enable the CiviMember Roles Sync module in the CiviCRM modules fieldset.
3. Click on the Save Configuration button.

In order to configure CiviMember Roles Sync, follow these steps:

1. Create one or more non-default Drupal roles.
2. Create and configure the CiviCRM membership types and statuses at Administer | CiviMember | Membership Types and Administer | CiviMember | Membership Status Rules (see Chapter 9, Growing Your Membership and Interacting with Members for more details).
3. In Drupal, navigate to Administer | Site Configuration | CiviMember Roles Sync.
4. Click on the Add Association Rule tab.
5. Select a CiviMember membership type, and select a Drupal role, in order to implement that every contact with a current status for that membership type has that Drupal role assigned to it.
6. Click on it to enable the appropriate statuses for both Current Status and Expired Status. If you are using the default CiviCRM membership status rules, you'll probably want to enable New, Current, and Grace for Current Status, and Expired, Pending, Cancelled, and Deceased for Expired Status. When the membership type for a contact is changed, these lists will be checked to determine whether the selected Drupal role should be assigned to or unassigned from the contact's Drupal account.
7. Click on Add association Rule.

Drupal – the CiviCRM OG Sync module

This module can ensure that the changes in the contacts in CiviCRM groups are reflected in associated Drupal organic groups. However, as the organic groups module does not provide an appropriate way to notify CiviCRM of changes in membership of organic groups, we find this module to be of limited usefulness.

Joomla! – the CiviAuthenticate plugin

The plugin is available at https://github.com/lcdservices/CiviCRM-CiviAuthenticate.

This Joomla! authentication plugin will replace the default Joomla! authentication plugin and provides the ability to perform the following tasks:

• Restricting login access based on membership status
• Redirecting expired members to a selected contribution page prefilled with their contact information
• Assigning the Joomla! access control groups to the user on login based on their membership status
• Assigning the Joomla! access control groups to the user on login based on their membership types

Joomla! – the CiviCRM Group Sync plugin

The plugin is available at https://github.com/lcdservices/CiviCRM-Group-Sync.

This component and plugin provides tools to create sync rules between the CiviCRM groups and Joomla! access control groups. Unlike the CiviAuthenticate plugin, these rules are applied whenever the person is added/removed from a synched group (in either Joomla! or CiviCRM). It does not sync with smart groups.

WordPress – the WordPress Member Sync plugin

The plugin is available at https://civicrm.org/extensions/civicrm-wordpress-member-sync.

This plugin keeps the WordPress users in sync with the CiviCRM memberships by granting either a role or capabilities to users with that membership.

There are undoubtedly other extensions available that enhance access control options, so do not assume that the preceding list is complete. Search the CiviCRM extension directory at the time you are implementing CiviCRM to review the current options available.

Dashboard

When you first log into your CMS and visit CiviCRM, you land on the Dashboard page. This page can be configured to display reports from your system in two columns. It is configured directly from this page using the Configure Your Dashboard button.

Clicking on it displays a visual representation of two columns and a list of available reports. Simply drag and drop from the available reports block to one of the two columns, and click on Done to save your settings.

The available reports, or dashlets, come from the main reports list (the Reports menu). You must configure a report to be available as a dashlet before it will appear in the dashboard configuration page. In addition, there are a few dashlets built specifically for that use (that is, they are not generated from the Reports tool). As an administrator, one that is particularly helpful for remaining aware of what's going on in the CiviCRM community is the CiviCRM news dashlet. It pulls a current feed from the CiviCRM blog, which will include information about new releases, new features, case studies, and other content of value to the community.

Navigation

Throughout our review of CiviCRM, we referenced the default location of tools and options in the horizontal navigation bar. However, you may add, move, remove, or rename the menu options in your site as you desire. For example, if your organization is a heavy user of activities (used to track communications with constituents) you may want to move the Activity Search and New Activity menu items to the main header.

You may modify the navigation menu by going to Administer | Customize Data and Screens | Navigation Menu. The menu is displayed as an expandable tree. Click on the New Menu Item button to create a new option, right-click on an existing option to modify it, and drag and drop items to move them to a new location.

While creating or editing an option, you enter the full URL for the page (including http://), unless it is an internal CiviCRM page, in which case you only need to include the path (for example, civicrm/dashboard?reset=1). This means that you may create menu options to non-CiviCRM locations, including those outside of your site. Menu items can be restricted based on the selected permissions.

After making modifications to the menu, a notification block will appear at the top of the page. Click on the Click here link to reload the menu and review your changes. Right-click on menu items to edit/delete/rename the item, and drag and drop the item to change its location in the tree.

Setting up Cron jobs

A Cron job is a time-based trigger configured on your hosting server. You set up the Cron jobs to automatically run scripts on a periodic basis. CiviCRM ships with a job processing script intended to be run via a Cron job, which triggers Scheduled Jobs found in Administer | System Settings.

But before we continue, let's ask the question—why do we even need processes running in our site that are triggered from a Cron job?

CiviCRM and the CMSes it integrates with are all written in PHP. PHP is a scripting language that is available on your web server and lets you create websites and applications. But websites, and the scripting languages they are built with, are reactive by nature. By this, we mean that they only operate when a user visits the site and begins to perform actions—clicking on a link, entering data in a form, running a search, and so on.

It's not unusual with an application such as CiviCRM to have certain maintenance or operational tasks that you want to run on a periodic basis—not simply in response to users' interactions with the site. This is where a Cron job becomes useful. It lets you trigger a PHP script (or other applications) on a time basis—such as every hour, or every Monday at a certain time, or twice a year.

To facilitate managing and processing these maintenance and operational tasks, CiviCRM provides the Scheduled Jobs tool, which lists a number of jobs that can be enabled, configured, and used to provide time-triggered functionality to your system. You only need to run a single Cron job to trigger the main scheduled jobs tool, and it will run the enabled jobs based on the frequency settings you've selected. There may be some circumstances where you choose to set up additional Cron jobs to trigger specific scheduled jobs, but that is not typically necessary.

Most hosting services will provide a control panel tool where you can schedule the Cron jobs using a graphical interface. You may also set up the Cron jobs by using the crontab command over SSH. The different flavors of Linux may have locations you can place scripts in, in order to have them executed periodically. For example, the files in /etc/cron.daily, /etc/cron.hourly, /etc/cron.monthly, and /etc/cron.weekly are executed at appropriate periods on the Debian servers. You should contact your hosting provider or seek other resources if you need guidance with creating the Cron jobs on your server.

Running the Scheduled Jobs tool via Cron

Depending on how you run the CiviCRM Cron script, you may need to retrieve the key value in your civicrm.settings.php file and create a user account permissioned to run administrative tasks on your site. The combination of a site key value and username/password help ensure that your Cron jobs can only be run by a valid source. The key value should be a 16-32 character alphanumeric string. You may generate one using online services, such as http://www.thebitmill.com/tools/password.html. Note that your key value should not contain any reserved special characters, including &, =, +, $, ?, %, ,, /, :, {, }, |, ', and #. Typically, this is not necessary, however, as CiviCRM will create a site key value when you first install the software.

The civicrm.settings.php file will generally be located in the following directory, depending on your CMS:

• Joomla!: /path_to_docroot/administrator/components/com_civicrm/
• Drupal: Either /path_to_docroot/sites/default/ or /path_to_docroot/sites/yourdomain.org/
• WordPress: /wp-content/plugins/civicrm/

Locate (or create) the site key definition line and add your configured key:

define( 'CIVICRM_SITE_KEY', 'your_key_value' );

The Cron job can be triggered in several ways. We will review them briefly.

PHP CLI

The PHP CLI method requires the username and password, but not the site key. It is run by triggering the Cron script with PHP. The command will look like the following line:

/path/to/php /path/to/civicrm/bin/cli.php -s site -u user -p password -e Job -a execute

The preceding command is explained as follows:

• -s site: This parameter is used to designate what site the script should be run on, which is most relevant for Drupal multisite implementations. Joomla! and WordPress users can set the value to default.
• -u user: This parameter is the username for the admin-level user you created.
• -p password: This parameter is the password for the admin-level user you created.
• -e Job: This parameter means the script should trigger the Scheduled Jobs tool.
• -a execute: This parameter indicates all enabled jobs should be run (or checked to see if they are ready to be run based on their frequency setting).

URL method

As the name suggests, the URL method lets you trigger the Cron from a URL. This can be used in your Cron settings with Wget or cURL, but also provides the flexibility to use an external Cron service, if desired. The command will look like the following line:

wget 'http://SITEROOT/PATH-TO-CIVICRM/bin/cron.php?name=username&pass=password&key=site-key'

The preceding command is explained as follows:

• name=username: This parameter is the username for the admin-level user you created
• pass=password: This parameter is the password for the admin-level user you created
• key=site-key: This parameter is the site key you retrieved from your civicrm.settings.php file

The URL method can be easily tested by simply constructing the URL with parameters and triggering it from your browser.

Drush method

Drush is a command-line utility used to manage the Drupal sites. If you are using Drupal and are comfortable working in the command line, take some time to learn about Drush—it is an incredibly powerful and useful tool to help with site management. To trigger the script with Drush, you would run the following command:

/usr/bin/drush -u 1 -r /path/to/drupal @sites civicrm-api job.execute auth=0 –y

Here, 1 is the ID of the user who will run the script.

WP-CLI method

WP-CLI is a command interface used to manage the WordPress sites.

/path/to/wp --user=username --url=<http://example.com> --path=</path/to/wp> civicrm api job.execute auth=0

The preceding command is explained as follows:

• --user=username: This parameter is the username admin-level user you created
• --url=<>: This parameter is the root website address
• --path=<>: This parameter is the path to CiviCRM

Other parameters

The preceding examples for running the jobs script are limited to when you would trigger all enabled scheduled jobs configured in the system. However, sometimes you want to trigger a specific scheduled job through the Cron system. In such a case, you would pass the script name and any parameters instead of the execute action. For example, if you wanted to run the update greeting job from the command line, you would use the following command:

php bin/cli.php -s site -u user -p password -e Job -a update_greeting --ct=Individual --gt=email_greeting

This command gives you much more control over when a job is run, which can be useful in some circumstances. Visit the Scheduled Jobs page to retrieve the job name and any parameters it will accept.

Managing scheduled jobs

As described earlier, the whole point of setting up the Cron job is to trigger the script that will run scheduled jobs from the interface. The goal, in most cases, is to set up a single Cron job, and then be able to manage the specific scheduled jobs and options through the interface. Visit Administer | System Settings | Scheduled Jobs to review the available jobs.

From this page, you can add a new job, view the logs for all jobs, configure specific jobs, view the logs for a specific job, run a job, or manage whether it is enabled or disabled. Some of the jobs will accept parameters, which will be detailed in the listing. When you configure the job, you will select how frequently it should be run (daily, hourly, or every time the Cron is run). Keep in mind that the frequency is determined in part by how you configured the Cron job.

Let's review the jobs and discuss their purpose and usage:

• Clean-up temporary data and files: Use this job to remove temporary data and files, and clear old data from cache tables. This can be run hourly to help maintain a clean system.
• Disable expired relationships: When working with relationships, there are two ways to mark one as inactive—you can explicitly disable it, or you can set an end date that is in the past. The latter is a very useful way to record when a person retires, leaves a company, or in some other way ends their relationship with the related contact. However, it can make it cumbersome to search active relationships if you must be concerned with both the disabled status and the relationship end date. This job will search for any relationships with an end date in the past, and mark them as disabled. This facilitates search and general management. It is sufficient to run this daily in most cases.
• Fetch bounces: Use this job to process your bounce e-mail inbox (discussed earlier in this chapter). The job will poll the account, retrieve any bounces, and determine how they should impact records (based on whether it was a soft or hard bounce). Based on that determination, bounced e-mails may be placed on hold, so they are not delivered to again. The job will also process any e-mail-based user actions, such as unsubscribe/resubscribe/optout. Bounced e-mails will follow rules configured in the system to put the e-mail account on hold, giving you the opportunity to review and correct the e-mail through the system.
• Geocode and parse addresses: CiviCRM's geocoding functionality generates a latitude and longitude value for addresses when they are created or edited. If you have existing addresses that are not geocoded or have recently imported new address records, use this script to batch-generate geocode values. Typically, you would only run this job on demand (manually), though if you are importing records on a regular basis it may be useful to configure it to run daily. The job will also parse address records, splitting the street address line into separate element fields. This job accepts parameters to process geocoding, address parsing, and optionally, set a start ID, end ID, and throttle. These optional parameters may be useful if you have a distinct set of records you want to process. Also keep in mind that the geocoding tool you are using (such as Google) may have a daily rate limit on how many records you can process.
• Mail reports: CiviCRM reports can be configured to be e-mailed on a periodic basis. For example, you may want a contribution report with the last week's summary of payments e-mailed to your staff bookkeeper every Monday morning, or your membership committee chair may want a membership overview report on a monthly basis. Use this job to initiate mailings and configure the frequency in which they should be sent. This job is one that you may consider creating a separate Cron job for, as it's unlikely you will have reports you want to have e-mailed on a daily, hourly, or more frequent basis. Note that you must provide several parameters—the format desired (CSV, print, or PDF) and the instance ID of the report to be sent. You can retrieve the latter from the URL when viewing the report.
• Process inbound e-mails: Earlier in this chapter, we walked through the configuration of an inbox for bounced e-mail processing. That same interface is used to configure an inbox used for inbound e-mail processing. E-mail activity processing will poll the inbox and attach e-mails as an activity record to existing contact records (if the e-mail is found in the system) or have new contact records created (if the e-mail is not in the system). This is a powerful way to automate communication tracking. For example, you might create a special e-mail account that your staff can optionally CC while sending e-mails to constituents, automatically attaching a record of the communication with the contact.
• Process pledges: This job is used by CiviPledge to process pledge reminders and update pledge statuses. When a user pledges donations to your organization, they determine the total value of their pledge and the frequency of their giving. If configured, this job will send pledge reminder e-mails as the date for their next donation draws near. It will also update the status of pledge commitments. It is sufficient to run this job once a day.
• Process survey respondents: If you are using the Campaign tools with Surveys and have configured an expiration period where reserved respondents are made available to other interviewers, you will want to run this job. It is responsible for reviewing the reservation date and releasing records that have been locked beyond the expiration period. It should typically be run hourly.
• Rebuild smart group cache: Smart groups are saved searches. When you pull the contacts of a smart group, CiviCRM reassesses and runs the criteria saved in the system, not a simple list of contacts added to the group, as with regular groups. However, running a query to generate the contacts in a group can be resource intensive at times. To alleviate performance issues, CiviCRM caches the contacts of a smart group, allowing them typically to be retrieved much more quickly. However, it's important to not let that cache get stale, or you risk having inaccuracies in your lists. This job will rebuild the smart group cache on a regular basis, allowing you to reliably pull an accurate list of contacts, without experiencing a performance hit through the interface. We recommend running this job hourly.
• Send scheduled mailings: This job is used by CiviMail to process scheduled (queued) broadcast e-mails. You must implement this job if you are using CiviMail. Generally, you will want to set it to every time the Cron is run. Also note that you may need to review the Mailer Setting options to throttle the number of e-mails sent per batch (Administer | CiviMail | Mailer Settings). Using the Cron frequency and spool period/batch limits, you can safely regulate how many e-mails are sent per hour (which may be limited by your hosting provider).
• Send scheduled reminders: The scheduled reminder tool lets you construct rules to trigger e-mails to selected recipients. For example, you could notify a certain person every time an activity of a certain type is created; or you could send an e-mail to event registrants three days before the event; or you could send out e-mail reminders to members a month before their membership expires. This job reviews those rules and sends the e-mails. It is generally sufficient to run it daily.
• Send scheduled SMS: Similar to the send scheduled mailings job, this will trigger any SMS mailings that have been scheduled and queued for delivery. Generally, you will set it to run every time the Cron is run.
• Update greetings and addresses: Every contact record has greeting values, which can be used for e-mails, mailings, and other forms of address, such as exports for merge letters. When records are created through the interface, the default greeting values are constructed (for example, Dear Mr. Smith). If you change the default greeting or recently imported contacts with no existing greeting values, run this script to update the value for all contacts in your database. It is generally not necessary to set this job to run regularly; you can simply trigger it manually as needed. Note that there are several variables that must be appended to the script in order to control what records and fields are impacted.
• Update membership statuses: This job is used by CiviMember to update membership statuses based on the status rules defined in your system. This is a very important job if you are a CiviMember user. Member status rules define when a membership record is considered new, current, in grace period, or expired (or any other statuses you define). This script must be run in order to process the records against those rules and update them when they pass from one status to the next. It is sufficient to run this once a day.
• Update participant statuses: This job is used by CiviEvent if you enable waitlisting or require an approval for event registration. Waitlisting is a function available for events that have an attendance cap. Once the registration for the event has filled up, users will be given the option of being added to the waitlist, and if the available space increases or existing registrants cancel, you can allow waitlisted people to register. The script handles participant status updates. The On waitlist statuses are moved to the Pending from waitlist when space becomes available for the event. If you configure an expiration period for an event, the Pending statuses will be changed to Expired. In each case, the job will send an e-mail notification to registrants, alerting them of the status change. This job should be implemented only if you plan to use waitlisting or require approval for event registration. In general, it will be sufficient to run this script once an hour, though if you have high demand events, you may want to run it every time the Cron is run. Note that in order to use the waitlisting and approval features, you must enable the relevant participant statuses found in Administer | CiviEvent | Participant Statuses.
• Validate e-mail address from mailings: When an e-mail bounces, the fetch bounces job will retrieve the e-mail from the configured inbox and try to determine whether it is a hard bounce, which should be put on hold immediately, or a soft bounce, which typically would have a three strikes rule—it must bounce three times before it's placed on hold. One example of a soft bounce is when the mailbox is full. In such a case, it's certainly possible the user will get around to cleaning out their inbox, free up space, and will be able to accept new mail. This job looks for e-mails that have an existing strike, checks to see if a subsequent mailing was delivered successfully, and if so, resets the strikes to zero. It is sufficient to run this daily, unless you often schedule multiple bulk e-mails for the same day, in which case, you'll want to run it more frequently.
• /path_to_docroot/administrator/components/com_civicrm/civicrm/bin/, and in Drupal, they will generally be located in /path_to_docroot/sites/all/modules/civicrm/bin/. Let's review each of the scripts that can be run through a Cron job:
• (select one, required): This script determines the contact type that will be updated.
• (select one, required): This script determines the greeting value that will be updated.

This selects the option value for the greeting or addressee format. This is optional, and would only be used if you want to populate the field with a greeting format different from the configured default one. This updates all the records using the default greeting format, even if a value already exists. This updates only those records that have a greeting ID set, but the value of which has not been constructed. This is useful for contact record imports that may have had the default ID assigned, but need to have the greeting text constructed.

PHP: CiviCRM reports can be configured to be e-mailed on a periodic basis. For example, you may want a contribution report with the last week's summary of payments e-mailed to your staff bookkeeper every Monday morning, or your membership committee chair may want a membership overview report on a monthly basis. Use this Cron job to initiate mailings and configure the frequency in which they should be sent. This Cron job works differently from others—you will configure a Cron job for each report instance for this Cron job, unlike the others. The parameters are passed in the Cron URL (in addition to the user, password, and key values), .../bin/CiviReportMail.php?sendmail=true&instanceId=3&reset=1. To determine the report's instanceId, visit the report and note the value after /instance/ in the URL.

This part of the configuration process is admittedly more technical than most other areas. However, if you take the time to understand the concepts, and seek assistance where necessary to wade through the technical pieces, you will find the resulting functionality very rewarding. As you can see from the list of jobs available, there are some very important functions offered.

As with anything else, be sure to test it thoroughly. The Cron jobs can generate logs that are sent to an e-mail address, which is a great way to ensure that they are functioning properly. You should also schedule periodic system reviews in order to confirm that your Crons are running as expected. The scheduled job logs are a good way to do a first round of analysis if you suspect things are not working correctly.

For additional information about scheduled jobs, visit the CiviCRM Wiki page: http://wiki.civicrm.org/confluence/display/CRMDOC/Managing+Scheduled+Jobs.