InfoQ

Article

Getting Started With SharePoint Web Services

Posted by Trent Swanson on Feb 06, 2008

Community
.NET
Topics
Web Services ,
Interop ,
Application Servers

SharePoint supports interoperability and remote operations through a set of web services, and by SharePoint I am referring to WSS 3.0 (Windows SharePoint Services) and MOSS 2007 (Microsoft Office SharePoint Server). There are number of approaches for programmatic access to SharePoint, such as the object model, web services, RPC over HTTP, as well as WebDAV, all of which have their benefits and their place depending on the needs of the application. The SharePoint RPC protocols can be useful and powerful and it certainly has its strengths, especially when it comes to adding content. The object model is much more robust and feature rich than the web services, but it does not offer support for remote operations, so in order to leverage the object model your code must be running on the SharePoint server, and you must be writing .NET code in order to leverage it. There are a number of different reasons for working with SharePoint data remotely, be it administrative scripts or the need to work with SharePoint content such as tasks in an application running on a client machine, or non-SharePoint applications in the enterprise. The SharePoint web services are built on top of the object model and expose a subset of features available in the object model, and allow for remote operations and the use of any language and platform that supports the consumption of web services. For most tasks the web services that ship in SharePoint are adequate, but you may find needed operations in the object model that have not been surfaced in the web services, which will require extending the web services with a custom web service implementation. In this article we will take a look at the web services available in WSS3 and MOSS 2007 out of the box, and how to consume them from Java and .NET.

Web Services Architecture Overview

I will not go in to the detailed differences between the WSS and MOSS, as there are a number of articles that do a very good job explaining this, only as needed and where related to the web services. Keep in mind that MOSS 2007 is built on top of WSS 3.0, which is also built on top of ASP.NET 2.0 and runs in IIS (Internet Information Server). A solid understanding of IIS and ASP.NET is important, and go long way in working with and understanding SharePoint. A majority of the SharePoint web services we will discuss are included in WSS 3.0, but there are a few additional ones that are included in MOSS 2007 to leverage the additional features that ship with it, such as the more advanced enterprise search features.

The SharePoint web services are implemented in ASP.NET Web Services (ASMX), and you will find the physical files for most of these web services in the "Microsoft Shared" directory under "web server extensions\12\ISAPI" typically located at "C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI". The admin web service for the Central Administration tool is located in the ADMISAPI folder for which the central administration console contains a virtual directory named "_vti_adm". When a SharePoint site is created it will contain a virtual directory named "_vti_bin" that points to this location. IIS does not contain any application or virtual directories for subwebs, they do however contain mapping to the _vti_bin virtual directory through SharePoint metadata and HttpModules.



For each service you will find a *.wsdl.aspx file that generates the service WSDL (Web Services Description Language), a *.disco.aspx providing the service discovery implementation, and the actual *.asmx endpoint files, most of which simply contain a page directive referencing the SharePoint assembly and type containing the actual implementation for the service.

Service File Type Description
*.wsdl.aspx (search.wsdl.aspx) File that generates the WSDL (Web Services Description Language) in SharePoint
*.disco.aspx (seach.disco.aspx) File that provides the discovery implementation
*.asmx (search.asmx) Web services endpoint, most of which contain a page directive referencing the SharePoint assembly and type containing the actual implementation for the service

Out of the Box Services

There are a number of web services implemented OOTB (out of the box) in SharePoint that will address most of the common and basic tasks, from administrative tasks to search and working with list data, and much more. Below is a listing of the SharePoint web services with an overview of the functionality exposed for your reference. A simple expanded list like this helps me in working with the web services giving me a quick view of the overall services and methods available.

Service Administration
(_vti_adm/Admin.asmx) Administrative methods for creating deleting sites and retrieving languages used in the deployment
  • CreateSite
  • DeleteSite
  • GetLanguage
  • RefreshConfigCache
Alerts (Alerts.asmx) Methods for working with SharePoint list item alerts
  • DeleteAlerts
  • GetAlerts
Authentication (Authentication.asmx) Client proxy that provides user authentication for sites that use forms-based authentication
  • Login (Used to authenticate)
  • Mode (Returns the authentication mode of the current site)
Copy (Copy.asmx) Methods to copy files between or within sites
  • CopyIntoItems (Copy document as byte array to location on server)
  • CopyIntoItemsLocal (Copy document from one location on the same server to another)
  • GetItem (Creates a byte array of a document that can be passed to theCopyIntoItems method)
Document Workspace (Dws.asmx) Methods for managing Document Workspace sites and data
  • CanCreateDwsUrl
  • CreateDws
  • CreateFolder
  • FindDwsDoc
  • GetDwsData
  • GetDwsMetaData
  • RemoveDwsUser
  • RenameDws
  • UpdateDwsData
Forms (Forms.asmx) Methods for returning forms that are used in the user interface when working with the contents of a list
  • GetForm
  • GetFormCollection
Imaging (Imaging.asmx) Methods to create and manager picture libraries
  • CheckSubwebAndList
  • CreateNewFolder
  • Delete
  • Download
  • Edit
  • GetItemsByIds
  • GetItemsXMLData
  • GetListItems
  • ListPictureLibrary
  • Rename
  • Upload
List Data Retrieval (DspSts.asmx) Perform queries against sites and list in SharePoint
  • Query (Performs queries against SharePoint lists and sites)
Lists (Lists.asmx) Methods for working with Lists and List Data
  • AddAttachment
  • AddDiscussionBoardItem
  • AddList
  • AddListFromFeature
  • ApplyContentTypeToList
  • DeleteAttachment
  • DeleteContentType
  • DeleteContentTypeXmlDocument
  • DeleteList
  • GetAttachmentCollection
  • GetList
  • GetListAndView
  • GetListCollection
  • GetListContentType
  • GetListContentTypes
  • GetListItemChanges
  • GetListItemChangesSinceToken
  • GetListItems
  • GetVersionCollection
  • UndoCheckout
  • UpdateContentType
  • UpdateContentTypesXmlDocument
  • UpdateContentTypeXmlDocument
  • UpdateList
  • UpdateListItems
Meetings(Meetings.asmx) Create and manage Meeting Workspace Sites
  • AddMeeting
  • AddMeetingFromICal
  • CreateWorkspace
  • DeleteWorkspace
  • GetMeetingInformation
  • GetMeetingWorkspaces
  • RemoveMeeting
  • RestoreMeeting
  • SetAttendeeResponse
  • SetWorkspaceTitle
  • UpdateMeeting
  • UpdateMeetingFromICal
People(People.asmx) Resolve and find Principals
  • ResolvePrincipals
Permissions (Permissions.asmx) Methods for working with permissions for a site or list
  • AddPermission
  • AddPermissionCollection
  • GetPermissionCollection
  • RemovePermission
  • RemovePermissionCollection
  • UpdatePermission
Directory Management(sharepointemailws.asmx) Methods for managing Active Directory e-mail distribution groups and their memberships
  • ChangeContactsMembershipInDistributionGroup
  • ChangeUsersmembershipInDistributionGroup
  • CreateContact
  • CreateDistributionGroup
  • DeleteContact
  • DeleteDistributionGroup
  • GetJobStatus
  • ModifyContact
  • ModifyDistributionGroup
  • RenameDistributionGroup
Site Data (SiteData.asmx) Methods that return metadata or list data from sites or lists
  • EnumerateFolder
  • GetAttachments
  • GetChanges
  • GetContent
  • GetList
  • GetListCollection
  • GetListItems
  • GetSite
  • GetSiteAndWeb
  • GetSiteUrl
  • GetURLSegments
  • GetWeb
Sites(Sites.asmx) Methods for returning information about the collection or site template
  • ExportWeb
  • GetSiteTemplates
  • GetUpdatedFormDigest
  • ImportWeb
Search(spsearch.asmx) Methods for searching via search services
  • Query
  • QueryEx
  • Registration
  • Status
Users & Groups(usergroup.asmx) Methods for working with users role definitions and groups
  • AddGroup
  • AddGroupToRole
  • AddRole
  • AddRoleDef
  • AddUserCollectionToGroup
  • AddUserCollectionToRole
  • AddUserToGroup
  • AddUserToRole
  • GetAllUserCollectionFromWeb
  • GetGroupCollection
  • GetList
  • GetListAndView
  • GetListCollection
  • GetGroupCollectionFromRole
  • GetGroupCollectionFromSite
  • GetGroupCollectionFromUser
  • GetGroupCollectionFromWeb
  • GetGroupInfo
  • GetRoleCollection
  • GetRoleCollectionFromGroup
  • GetRoleCollectionFromUser
  • GetRoleCollectionFromWeb
  • GetRoleInfo
  • GetRolesAndPermissionsForCurrentUser
  • GetRolesAndPermissionsForSite
  • GetUserCollection
  • GetUserCollectionFromGroup
  • GetUserCollectionFromRole
  • GetUserCollectionFromSite
  • GetUserCollectionFromWeb
  • GetUserInfo
  • GetUserLoginFromEmail
  • RemoveGroup
  • RemoveGroupFromRole
  • RemoveRole
  • RemoveUserCollectionFromGroup
  • RemoveUserCollectionFromRole
  • RemoveUserCollectionFromSite
  • RemoveUserFromGroup
  • RemoveUserFromRole
  • RemoveUserFromSite
  • RemoveUserFromWeb
  • UpdateGroupInfo
  • UpdateRoleDefInfo
  • UpdateRoleInfo
  • UpdateUserInfo
Versions (Versions.asmx) Methods for working with file versions
  • DeleteAllVersions
  • DeleteVersion
  • GetVersions
  • RestoreVersion
Views(Views.asmx) Methods for working with list views
  • AddView
  • DeleteView
  • GetViewCollection
  • GetViewHtml
  • UpdateView
  • UpdateViewHtml
  • UpdateViewHtml2
Web Part Pages(WebPartPages.asmx) Methods for working with Web Part Pages
  • AddWebPart
  • AddWebPartToZone
  • AssociateWorkflowMarkup
  • ConvertWebPartFormat
  • DeleteWebPart
  • ExecuteProxyUpdates
  • FetchLegalWorkflowActions
  • GetAssemblyMetaData
  • GetBindingResourceData
  • GetCustomControlList
  • GetDataFromDataSourceControl
  • GetFormCapabilityFromDataSourceControl
  • GetSafeAssemblyInfo
  • GetWebPart
  • GetWebPart2
  • GetWebPartCrossPageCompatibility
  • GetWebPartPage
  • GetWebPartPageConnectionInfo
  • GetWebPartPageDocument
  • GetWebPartProperties
  • GetWebPartProperties2
  • RemoveWorkflowAssociation
  • RenderWebPartForEdit
  • SaveWebPart
  • SaveWebPart2
  • ValidateWorkflowMarkupAndCreateSupportObjects
Webs(Webs.asmx) Methods for working with sites and subsites
  • CreateContentType
  • CustomizeCss
  • DeleteContentType
  • GetActivatedFeatures
  • GetAllSubWebCollection
  • GetColumns
  • GetContentType
  • GetContentTypes
  • GetCustomizedPageStatus
  • GetListTemplates
  • GetWeb
  • GetWebCollection
  • RemoveContentTypeXmlDocument
  • RevertAllFileContentStreams
  • RevertCss
  • RevertFileContentStream
  • UpdateColumns
  • UpdateContentType
  • UpdateContentTypeXmlDocument
  • WebUrlFromPageUrl
  • MOSS Search (Search.asmx) Methods for searching via MOSS (Microsoft Office SharePoint Server) Search services, which also includes a method to retrieve the managed search properties
    • GetSearchMetadata (Search managed properties)
    • Query
    • QueryEx
    • Registration
    • Status

    Consuming the Query Services

    Now that we have an understanding of the SharePoint web services, there is nothing like kicking the tires and taking a ride after reading through the owner's manual. We don't have time here to dive deep in to all of the web services, and will save the others for the next article. For some reason a number of the web services in SharePoint either take or return an XML Node or an XML encoded string, and unfortunately without a typed schema the tools are unable to infer what this should be and generate those nice proxy objects for us. I have seen a number of implementations using string formatting with the web services to address the lack of a typed schema, and have searched long and far for a schema that describes the XML passed and returned from these services. I ended up creating queryresponse.xsd which was inferred from a combination of the documentation and the xml passed in and out of these services, allowing me to generate the class representation using xsd.exe for .NET users and JAXB for java users.

    Note: There are two different search web services, as discussed previously, search.asmx for MOSS and spsearch.asmx for WSS. These services are very similar in their schema, but attempting to call spsearch.asmx on a machine running MOSS will throw an exception due to the fact that the WSS indexing and query services have been disabled in favor of the more advanced search features in MOSS.

    Example using .NET

    Consuming the SharePoint web services from .NET is rather simple and straight forward, you simply need to add a web reference and use the generated proxy. We will add a step to leverage the XSD we created to generate a set of classes representing a query and response that we will use to serialize the XML sent and deserilize the XML returned.

    Configure Visual Studio 2005 to add XSD.exe to the menu

    Visual studio includes a tool called XSD.exe that can be used to generate .NET types from an XSD. This is however a command line tool and I like to add a menu option to visual studio to do this from the IDE. If you have already done this or have some other add-in to do something similar you can skip this step.

    1. Open the "External Tools" Dialog in the Visual Studio "Tools" menu
    2. Click "Add"
    3. Set the necessary properties and click "Ok"
    Property Value
    Command XSD >> CSharp
    Title C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin\xsd.exe
    Arguments $(ItemPath) /c
    Initial Directory $(ItemDir)
    Use Output Window Checked

    Download the XSD files

    Download the XSD files from here which will be used to generate the proxy classes used to serialize and de-serialize the search requests and responses. Unpack the downloaded file to a working directory; this file contains the XSD along with the completed project.

    Create a Project and Generate a .NET type from the XSD Now that we have the XSD we will create a Windows Forms project and generate a .NET type form the downloaded XSD. After opening up Microsft.Search.Query.xsd, go to the tools menu and select the XSD menu item that we created in the previous step to generate the Microsoft_Search_Query.cs file.

    Add a web reference

    We are going to start by adding a web reference to the search web service, by right-clicking on the "references" node for our project in the "solution explorer" window. There we will select "Add Web Reference", enter "http://<server_name>/_vti_bin/spsearch.asmx" in the URL textbox and then click "Go". You may be prompted to login to the SharePoint server and you should now have a list of Methods supported by this service. Add "SPSearch" for the "Web Reference Name" and click "Add Reference".

    Subclass the generate proxy

    To simplify things even further, we will subclass the generated proxy with our own so that we can wrap the QueryService to hide the validation, serialization, and de-serialization of the XML passed and returned in the Query method using our class we generated from the XSD.

    Create the application to call the service

    Now we simply instantiate a query class and QueryService class, and then pass the query object to the Query method on the QueryService class.

    SPSearch.QueryService search = new SPSearch.QueryService();
    Microsoft_Query_Request request = new Microsoft_Query_Request();
    search.Query(request);

    Example using Java

    Consuming the SharePoint web services from Java is not much different, the fundamental steps are the same, generate our class from the provided XSD, generate a web services proxy, and build the application. Most of the challenges you will face in consuming SharePoint web services from java are with authentication and the use of DataSet's in the SharePoint web services. There are a number of Java IDEs available to simplify working with web services, but since I primarily work with .NET these days I will show an example using the SDK and in this case leave the choosing of an IDE to you.

    Note: This example was built and tested using J2SE 6.0 Update 3

    Java SharePoint Sample Project Directory and Root Files

    The full project and generated files are included along with couple of batch files to build and run the sample. You will need to modify "xjc-build.bat" and "build-run.bat" with the home directory for your JDK bin folder to run this, and edit wsspsample/Main.java with your server settings (endpoint, username, and password) before building and running.

    Import the Service WSDL

    I start by retrieving the WSDL from my SharePoint server to my development machine for a couple reasons. We will need to modify any WSDL that uses ADO.NET Data Sets, so as to work properly with the tools. Also, access to the WSDL on the SharePoint server typically requires authentication and the Java generated proxy requires the WSDL in the constructor, making authentication a bit of problem for me.

    Retrieve the Search WSDL

    For Windows SharePoint Services 3.0 search service open up Internet Explorer and navigate to Error! Hyperlink reference not valid. then from File|Save As..., save the page to the working directory as spsearch.wsdl. If you are working with MOSS 2007 then you will use Error! Hyperlink reference not valid.

    Modify the Search WSDL

    The QueryEx method as well as the MOSS 2007 GetSearchMetaData method returns ADO.NET Data Sets. An ADO.NET Data Set is dynamically bound and is represented in WSDL with a cyclic reference to the WSDL schema attribute, which causes some problems for us with the JAXB wsimport tool. I found some workarounds that worked well with previous versions of the Java JDK, but these did not work with the version I was using. Instead I chose to modify the WSDL as follows by opening it up in notepad and searching for instances of "s:schema" to modify where it was used as a ref in a schema "element" element, and remove the cyclic reference and leaving the schema "any" element.

    <s:sequence>
    <s:element ref="s:schema"/>
    <s:any/>
    </s:sequence>

    Was change to the following:

    <s:sequence>
    <s:any minOccurs="0" maxOccurs="unbounded"/>
    </s:sequence>

    Import the WSDL

    Now that we have our WSDL locally and it has been modified, we can use JAXB wsimport.exe tool to generate the proxy classes.

    wsimport -p wsspsample.webref.spsearch -keep spsearch.wsdl
    wsimport -p wsspsample.webref.search -keep search.wsdl

    We use the -keep flag with the import to keep the java code so that we can tweak the generated code for our application. The tools generate code with an absolute path, so if you are planning on moving the application to a different directory and storing the WSDL with the application we will need to modify "webref\search\QueryService.java" url and build it ourselves. To do this we simply remove the absolute path to the WSDL and use a relative path instead.

    url = new URL("file:search.wsdl");

    Generate Request Response Classes

    To generate the Request and Response classes you are going to need to use the four "Microsoft.Search" XSDs that I have created and the JAXB xjc.exe tool. In the following we simply suppress generation of package level annotation and specify a target package, and the tool creates our classes for us.

    xjc -npa -p wsspsample.xom.query -d . Microsoft.Search.Query.xsd
    xjc -npa -p wsspsample.xom.response -d . Microsoft.Search.Response.xsd

    Use the Generated Classes

    We have an class representation of the Query and Response packets passed to and returned form the QueryService Query method, as well as a Query service proxy. Now it's time to put that to use in an application. The first thing we need to do is create a QueryService instance, the constructor for this object will go and retrieve the local copy of our WSDL and load settings. From the QueryService instance we retrieve a QueryServiceSoap instance via the getQueryServiceSoap method. Note that you will need to hang on to and use the QueryServiceSoap reference as we do with the qsp variable, since the getQueryServiceSoap method seems to return a new instance. We can then modify the endpoint address here to something other than what is contained in the WSDL to make our application more dynamic allowing it to connect to other SharePoint sites and SubSites other than that set in the WSDL.

    QueryService qs = new QueryService();
    QueryServiceSoap qsp = qs.getQueryServiceSoap();

    BindingProvider bp = (BindingProvider)qsp;
    //bp.getRequestContext().put(BindingProvider.USERNAME_PROPERTY, "Administrator");
    //bp.getRequestContext().put(BindingProvider.PASSWORD_PROPERTY, "pass@word1");
    bp.getRequestContext().put(BindingProvider.ENDPOINT_ADDRESS_PROPERTY,
    "http://barbie/_vti_bin/spsearch.asmx");

    Now we create methods to serialize the query and deserlialize the response packets.

    public static String SerializeQuery(QueryPacket qp)
    {
    try
    {
    JAXBContext jc =JAXBContext.newInstance("wsspsample.xom.query");
    Marshaller ma = jc.createMarshaller();
    ma.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT,Boolean.TRUE);
    ByteArrayOutputStream os = new ByteArrayOutputStream();
    ma.marshal(qp, os);
    return os.toString();
    }
    catch(JAXBException ex)
    {
    return "";
    }
    }
    public static ResponsePacket DeserializeResponse(String s)
    {
    try
    {
    JAXBContext jc = JAXBContext.newInstance("wsspsample.xom.response");
    Unmarshaller um = jc.createUnmarshaller();
    StreamSource source = new StreamSource(new StringReader(s));
    return (ResponsePacket)um.unmarshal(source);
    }
    catch (JAXBException ex)
    {
    return new ResponsePacket();
    }
    }

    From here we simply create a QueryPacket and necessary classes, set our values and call the service.

    // Create a Search Query Packet Object
    QueryPacket qp = new QueryPacket();
    QueryType qt = new QueryType();
    ContextType ct = new ContextType();
    QueryTextType ctt = new QueryTextType();

    ct.setQueryText(ctt);
    qt.setContext(ct);
    qp.setQuery(qt);

    // Set search values
    ctt.setValue("sharepoint");
    ctt.setType("STRING"); //This is the default - not necessary
    ctt.setLanguage("en-us"); //This is the default - not necessary

    // Call the web service query
    String sResponse = qsp.query(SerializeQuery(qp));

    // Deserialize the response
    ResponsePacket resp = DeserializeResponse(sResponse);

    Build and Run the Application

    We have our generated web services proxy and object model for query request and response, as well as the application that leverages them, now we simply need to cross our fingers, build, run, and test this. You will need a SharePoint server setup and will want to set the correct endpoint address used in Main.java to your test SharePoint server as well as the credentials if necessary.

    References and Links

    Windows SharePoint Services 3.0 Web Services Reference.

    21 comments

    Watch Thread Reply

    WSRP by Aeh. ABID Posted Feb 11, 2008 4:14 AM
    Java Sample Code Missing by Steve Reynolds Posted Feb 13, 2008 6:59 AM
    Re: Java Sample Code Missing by Diana Plesa Posted Mar 10, 2008 5:17 AM
    XSD files by Garth Sweet Posted Mar 5, 2008 8:16 AM
    Re: XSD files by Diana Plesa Posted Mar 10, 2008 5:17 AM
    Re: XSD files by tamer morsi Posted Mar 12, 2008 10:15 AM
    Re: XSD files by Floyd Marinescu Posted Mar 18, 2008 7:59 AM
    Re: XSD files by tamer morsi Posted Mar 18, 2008 9:39 AM
    Re: XSD files by aladin sohaili Posted Apr 11, 2008 3:49 PM
    Re: XSD files by Markus Schmidt Posted Dec 1, 2009 3:43 PM
    Re: XSD files by Peter Trein Posted Jan 28, 2010 4:01 PM
    Re: XSD files by ian smith Posted Feb 11, 2009 3:52 AM
    Re: XSD files by HariKrishna Maharshi Posted Apr 16, 2009 1:56 PM
    NTLM Authentication by Steve Reynolds Posted Mar 12, 2008 9:19 AM
    Re: NTLM Authentication by tamer morsi Posted Mar 12, 2008 1:10 PM
    Re: NTLM Authentication by Kathryn Steel Posted Aug 1, 2008 2:43 AM
    Re: NTLM Authentication by Kathryn Steel Posted Aug 1, 2008 2:44 AM
    Re: NTLM Authentication by Ken Nagel Posted Nov 21, 2008 5:03 PM
    Re: NTLM Authentication by Dilip Kumar Aerroju Posted Feb 6, 2009 5:40 AM
    org.apache.axis2.AxisFault: Transport error: 401 Error: Unauthorized by Stefaan Somers Posted Mar 1, 2009 12:46 PM
    Re: XSD files by HariKrishna Maharshi Posted Apr 16, 2009 11:03 AM
    1. Back to top

      WSRP

      Feb 11, 2008 4:14 AM by Aeh. ABID

      Hi,
      does Sharepoint support WSRP ? since I'd found any official resources about sharepoint interop through WSRP.

    2. Back to top

      Java Sample Code Missing

      Feb 13, 2008 6:59 AM by Steve Reynolds

      I do not see a link to download the aforementioned Java Sample Project. Am I just missing it?

      Thanks,
      Steve

    3. Back to top

      XSD files

      Mar 5, 2008 8:16 AM by Garth Sweet

      Very helpful article!
      In the article you mention under the "Download the XSD files" section that we can "Download the XSD files from <here>...." but <here> isn't actually a link to anything. Can you point me to the XSD files?


      </here></here>

    4. Back to top

      Re: Java Sample Code Missing

      Mar 10, 2008 5:17 AM by Diana Plesa

      Hello Steve

      I've just added the download link.

      diana

    5. Back to top

      Re: XSD files

      Mar 10, 2008 5:17 AM by Diana Plesa

      Hi Garth

      I've added a link for the zip file containing the XSD files

      Diana

    6. Back to top

      NTLM Authentication

      Mar 12, 2008 9:19 AM by Steve Reynolds

      Diana,
      Thanks for posting the code. I have been trying to connect to the SharePoint Search Web Service via Axis2, but can not get it to work with NTLM Authentication. I am getting a similar error with your method: Exception in thread "main" com.sun.xml.internal.ws.client.ClientTransportException: request requires HTTP authentication: Unauthorized
      I was wondering if any has tried this with NTLM Authentication. The SharePoint server I am connecting to has Basic Authentication turned off because it sends the credentials in plain text. Note: I provided a valid administrator username and password in Main.java. Looking at the BindingProvider API, I don't see any other properties that I need to provide. I'm out of ideas.

    7. Back to top

      Re: XSD files

      Mar 12, 2008 10:15 AM by tamer morsi

      Thanks for uploading the docs,

      The xsd.zip file; the article says: this file contains the XSD along with the completed project.
      but there are no other files in the xsd.zip,
      are their any missing files ?
      thanks again

    8. Back to top

      Re: NTLM Authentication

      Mar 12, 2008 1:10 PM by tamer morsi

      Hey,

      check out:
      derekjmiller.spaces.live.com/blog/cns!A77124B9D...

      // Setup Authentication parameters
      String username = "Derek.Miller";
      String password = "mypassword";
      String host = "moss.syddev01.mymachine.com";
      // NOTE: Important to ensure fully qualified domain name is used
      String domain = "syddev01";
      String realm = "syddev01";
      int port = 80;

      // Configure authenticator
      HttpTransportProperties.Authenticator auth =
      new HttpTransportProperties.Authenticator();
      auth.setUsername(username);
      auth.setPassword(password);
      auth.setDomain(domain);
      auth.setHost(host);

      // Ensure NTLM authentication used
      List authPrefs = new ArrayList(1);
      authPrefs.add(AuthPolicy.NTLM);
      auth.setAuthSchemes(authPrefs);


      // Get stub
      ListsStub lists =
      new ListsStub("http://moss.syddev01.mymachine.com/_vti_bin/Lists.asmx");

      // Set authenticator
      lists._getServiceClient().getOptions().setProperty
      (org.apache.axis2.transport.http.HTTPConstants.AUTHENTICATE, auth);
      // Get request and execute
      GetListCollection req = new GetListCollection();
      GetListCollectionResponse res = lists.GetListCollection(req);
      DisplayResult(req, res);

    9. Back to top

      Re: XSD files

      Mar 18, 2008 7:59 AM by Floyd Marinescu

      tamer, there’s another link in the article right below the image Java SharePoint Sample Project Directory and Root Files that contains the other archive (The full project and generated files are included along with couple, ...)

    10. Back to top

      Re: XSD files

      Mar 18, 2008 9:39 AM by tamer morsi

      Thanks but there should be another c# class, that is missing : Microsoft_Query_Request.c
      This class is different from the generated xsd classes & is a wrapper class that the author refers to & the author says it is included in the xsd zip file ...
      though i maybe wrong on this;
      either way thnks

    11. Back to top

      Re: XSD files

      Apr 11, 2008 3:49 PM by aladin sohaili

      Hi Guys,
      Thanks for the Article and for the different posts.

      i modified the absolute path for the WSDL,and i modified the Main.java to point to my sharepoint server and to use the correct credentials.

      i'm trying to run the code, i'm having this exception:

      com.sun.xml.ws.client.ClientTransportException: request requires HTTP authentication: Unauthorized
      at com.sun.xml.ws.transport.http.client.HttpClientTransport.checkResponseCode(HttpClientTransport.java:212)
      at com.sun.xml.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:149)
      at com.sun.xml.wss.jaxws.impl.SecurityClientPipe.process(SecurityClientPipe.java:208)
      at com.sun.xml.ws.api.pipe.helper.PipeAdapter.processRequest(PipeAdapter.java:115)
      at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:595)
      at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:554)
      at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:539)
      at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:436)
      at com.sun.xml.ws.client.Stub.process(Stub.java:248)

      Thanks a lot for the help.

    12. Back to top

      Re: NTLM Authentication

      Aug 1, 2008 2:43 AM by Kathryn Steel

      I have tried this in Java and failed to get it to work with our server.
      I can get basic-authentication working but we need to get at least basic authentication with SSL or NTLM working.
      For SSL I get an
      java.lang.RuntimeException: Object reference not set to an instance of an object.
      Which suggest an object not populated that .NET is expecting.
      Code:
      I changed the URLS to be https:// for the ListsStub()


      Authenticator auth = new Authenticator();
      ArrayList authSchemes = new ArrayList();
      authSchemes.add(AuthPolicy.BASIC);
      auth.setAuthSchemes(authSchemes);
      auth.setAllowedRetry(false);

      auth.setUsername("doman\\name");
      auth.setPassword("password");
      auth.setPreemptiveAuthentication(true);

      copyStub._getServiceClient().getOptions().setProperty(HTTPConstants.AUTHENTICATE, auth);


      With NTLM I get Unable to respond to any of these challenges: {basic=Basic realm="melwwsinr08-basicauth"}
      org.apache.axis2.AxisFault: Transport error: 401 Error: Unauthorized

      code
      HttpTransportProperties.Authenticator auth = new HttpTransportProperties.Authenticator();
      auth.setUsername("ksteel");
      auth.setPassword("object123#");
      auth.setDomain("nt3gwm01");
      auth.setHost("10.16.224.86");
      auth.setRealm("");

      // Ensure NTLM authentication used
      List authPrefs = new ArrayList(1);
      authPrefs.add(AuthPolicy.NTLM);
      auth.setAuthSchemes(authPrefs);
      //
      copyStub._getServiceClient().getOptions().setProperty(HTTPConstants.AUTHENTICATE, auth);

      Has anyone got working for SSL or can help further with NTLM ?

    13. Back to top

      Re: NTLM Authentication

      Aug 1, 2008 2:44 AM by Kathryn Steel

      Sorry I should also say for SSL I have set up
      System.setProperty("javax.net.ssl.keyStore", "c:/client.keystore");
      System.setProperty("javax.net.ssl.keyStorePassword", "test12");
      System.setProperty("javax.net.ssl.keyStoreType", "JKS"); // JKS or PKCS12 ?

      System.setProperty("javax.net.ssl.trustStore", "c:/client.keystore");
      System.setProperty("javax.net.ssl.trustStorePassword", "test12");
      System.setProperty("javax.net.ssl.trustStoreType", "JKS"); // JKS or PKCS12 ?
      System.setProperty("javax.net.debug", "ssl");
      And I can see the SSL debug happenning and the application data content type 23 going over the wire encrypted but it gets an error. as I said saying Object reference not set to an instance of object coming back.

    14. Back to top

      Re: NTLM Authentication

      Nov 21, 2008 5:03 PM by Ken Nagel

      Kathryn
      Did you ever get your answer? My guys are stuck here as well, and cannot get through NTLMV2 auth

    15. Back to top

      Re: NTLM Authentication

      Feb 6, 2009 5:40 AM by Dilip Kumar Aerroju

      PermissionsStub permissionsStub = new PermissionsStub("http://apstra-87lg198t:44894/_vti_bin/Permissions.asmx");
      permissionsStub._getServiceClient().getOptions().setProperty(org.apache.axis2.transport.http.HTTPConstants.AUTHENTICATE, auth);
      GetPermissionCollection req1 = new GetPermissionCollection ();
      req1.setObjectName("");
      req1.setObjectType("web");
      GetPermissionCollectionResponse res1 = PermissionsStub.GetPermissionCollection(req1);

      UserGroupStub groupStub = new UserGroupStub("http://apstra-87lg198t:44894/_vti_bin/usergroup.asmx");
      groupStub._getServiceClient().getOptions().setProperty(org.apache.axis2.transport.http.HTTPConstants.AUTHENTICATE, auth);
      GetRoleCollectionFromUser collectionFromUser = new GetRoleCollectionFromUser();
      collectionFromUser.setUserLoginName("administrator");
      GetRoleCollectionFromUserResponse collectionFromUserResponse = groupStub.GetRoleCollectionFromUser(collectionFromUser);

      my problem is that very time authenticate for permission and userinfo. is there any way authenticate only once

    16. Back to top

      Re: XSD files

      Feb 11, 2009 3:52 AM by ian smith

      Does anyone know how the four XSD files were generated? Were they reverse engineered, or is this information available in a public place? I'm looking to interoperate with several sharepoint services using Java/JAXB.

    17. I'm pulling my hairs out for the moment. I've written some code, to get a list of sharepoint-lists. For some sharepoint sites, it works without any problem. But for other I always get the error : org.apache.axis2.AxisFault: Transport error: 401 Error: Unauthorized.

      There are 2 important questions I have :
      - how can I check which authentication mechanism is used (NTLM or BASIC)
      - can I get a more clear response, indicating me where the problem lies. Unauthorized is a bit too general. Moreover with the same user-name & password I can open the sharepoint URL for the list webservices.

      If anyone wants I can send the jar-file so he can try out for himself. I create a test-sharepoint side, where you can work on.

      Any help would be most welcome

    18. Back to top

      Re: XSD files

      Apr 16, 2009 11:03 AM by HariKrishna Maharshi

      Very useful article. Thanks.
      Also, can you explain, how these XSD files were generated? Or can we get them from sharepoint server?
      I wanted to interact with remaining services from the sharepoint server (from java).

    19. Back to top

      Re: XSD files

      Apr 16, 2009 1:56 PM by HariKrishna Maharshi

      Here is the link for Query schema on msdn site:

      msdn.microsoft.com/en-us/library/aa193736(office.11).aspx

    20. Back to top

      Re: XSD files

      Dec 1, 2009 3:43 PM by Markus Schmidt

      Hi Aladin

      I had the same problem and solved it by enabling Basic Authentication for the WebApplication in the central administration (Application Management -> Authentication Providers).

      Hope that helps,
      Markus

    21. Back to top

      Re: XSD files

      Jan 28, 2010 4:01 PM by Peter Trein

      has anyone tried connecting with WSS using Java?
      This code:
      BindingProvider bp = (BindingProvider)qsp;
      //bp.getRequestContext().put(BindingProvider.USERNAME_PROPERTY, "Administrator");
      //bp.getRequestContext().put(BindingProvider.PASSWORD_PROPERTY, "pass@word1");
      bp.getRequestContext().put(BindingProvider.ENDPOINT_ADDRESS_PROPERTY,
      "http://barbie/_vti_bin/spsearch.asmx");

      doesn't work with ListsSoap.:/ I get error 401.

      Anyone can help?

    Educational Content

    Brian Marick on 4 Challenges and 5 Guiding Values of Agile Software Development

    Brian Marick takes us through a quick tour of the most important values and challenges to adopting Agile successfully (they aren't the typical challenges and values we hear in the community).

    Are You a Software Architect?

    The line between development and architecture is tricky. Does it exist at all? Is an ivory tower actually needed? There's a balance in the middle, but how do you move from developer to architect?

    Agile – A Way of Life and Pragmatic Use of Authority

    The word 'authority' sometimes produces an allergic response in hard-line agilists. Freedom and authority – both are bad if misused and both are good if used in right spirit for a noble cause.

    Getting Started with Grails, Second Edition

    "Getting Started with Grails" brings you up to speed on this modern web framework. Companies as varied as LinkedIn, Wired, and Taco Bell are all using Grails. Are you ready to get started as well?

    Using ITIL V3 as a Foundation for SOA Governance

    Those familiar with only ITIL V2 often scoff at the thought that ITIL could serve as a governance framework for SOA. With ITIL V3, the focus of the framework shifted towards service-orientation.

    Adrian Colyer on AspectJ, tc Server and dm Server

    SpringSource CTO Adrian Colyer discusses AspectJ, SpringSource's dm Server and tc Server products, OSGi and Scrum.

    Adam Wiggins on Heroku

    Heroku's Adam Wiggins talks about Rails, Background Jobs, Add-Ons, Ruby, and how Heroku manages to work around Ruby's inefficiencies using Erlang and other languages.

    SOA as an Architectural Pattern: Best Practices in Software Architecture

    For Grady Booch the foundation of a good architecture is patterns, SOA being just one of many patterns. In this Second Life presentation, Booch attempts to bring more clarity on what architecture is.