Problem with nuget upload to symbol server when using RG

Dec 13, 2011 at 12:23 PM
Edited Dec 13, 2011 at 12:23 PM

I've just set up my RazorGenerator enabled project to push to Nuget using TeamCity. If I try to include symbols, and the automatic upload to SymbolServer, I get the following error

 

 

[13:17:50]  [push] Pushing FormFactory 1.1.18 to the symbol server (http://nuget.gw.symbolsource.org/Public/NuGet)...
[13:17:53]  [push] Package submission failed:
[13:17:53]  [push] System.InvalidOperationException: Source file c:\TeamCity\buildAgent\work\19471dda69268f25\FormFactory\Views\views\shared\formfactory\Form.Actions.cshtml not found for image file FormFactory.
[13:17:53]  [push]   at SymbolSource.Processing.Tools.PackageValidator.Validate (IDirectoryInfo directoryInfo) [0x00000] in :0 
[13:17:53]  [push]   at SymbolSource.Server.Management.WebService.CreateJob (SymbolSource.Server.Management.Caller caller, System.Byte[] zipContent) [0x00000] in :0 
[13:17:53]  [push]  - mismatched DLL/PDB files or missing source files (above message should detail which)
[13:17:53]  [push]  - general connectivity errors or unexpected behaviour of SymbolSource servers

That file is set as Content with RazorGenerator as it's Custom Tool

If I remove RazorGenerator from my project, the Nuget package deploys successfully. Not sure if it's a problem with RG or how TeamCity deploys Nuget, but I thought you guys would be the best ones to ask.

Cheers,

Harry

Feb 2, 2012 at 3:29 PM

*bump*

Coordinator
Feb 2, 2012 at 3:51 PM

Sorry, didn't see your post from before. Does the path look ok to you (it has Views repeated twice)? It might be the case that the source path pragmas that we generate aren't correct and SymbolSource is trying to validate it.

Feb 2, 2012 at 4:25 PM

Thats it I think, only one views directory, the real path is C:\TeamCity\buildAgent\work\19471dda69268f25\FormFactory\Views\Shared\FormFactory\Form.Actions.cshtml

May 10, 2012 at 2:44 PM
Edited May 10, 2012 at 2:44 PM

Hi I am still having a problem with this.

The problem occurs when my compiled helper stored in Noodles.Web/Helpers/BootstrapHelper.cshtml is compiled and the PDB is uploaded to symbolsource:

 

 
[15:35:32][Step 3/3] push: Publish package Noodles.Web.1.1.234.nupkg (9s)
[15:35:32][push] NuGet command: C:\TeamCity\buildAgent\tools\NuGet.CommandLine.1.6.0.nupkg\tools\NuGet.exe push C:\TeamCity\buildAgent\work\67dc14c75774fd53\Noodles.Web.1.1.234.nupkg %%teamcity_nuget_api_key_1336660532666%%
[15:35:32][push] Starting: C:\TeamCity\buildAgent\temp\agentTmp\custom_script6908978353646133926.cmd
[15:35:32][push] in directory: C:\TeamCity\buildAgent\work\67dc14c75774fd53
[15:35:33][push] Pushing Noodles.Web 1.1.234 to the NuGet gallery (https://www.nuget.org)...
[15:35:38][push] Your package was pushed.
[15:35:38][push] Pushing Noodles.Web 1.1.234 to the symbol server (http://nuget.gw.symbolsource.org/Public/NuGet)...
[15:35:41][push] Failed to process request. 'Package submission failed: Exception: Source file c:\TeamCity\buildAgent\work\67dc14c75774fd53\helpers\BootstrapHelper.cshtml not found in package. Source file is declared in Noodles.Web.pdb.. See http://www.symbolsource.org/Public/Home/Help for possible reasons. Fiddler may help diagnosing this error if your client discards attached detailed information.'.
[15:35:41][push] The remote server returned an error: (506) Package submission failed: Exception: Source file c:\TeamCity\buildAgent\work\67dc14c75774fd53\helpers\BootstrapHelper.cshtml not found in package. Source file is declared in Noodles.Web.pdb.. See http://www.symbolsource.org/Public/Home/Help for possible reasons. Fiddler may help diagnosing this error if your client discards attached detailed information...
[15:35:42][push] Process exited with code 1
[15:35:42][Step 3/3] Step NuGet Publish failed

 

The file actually exists at C:\TeamCity\buildAgent\work\67dc14c75774fd53\Noodles.Web\Helpers\BootstrapHelper.cshtml, so the PDB mistakenly thinks it is one directory up. I thought this was maybe because my project is a web project, so I changed it to output to bin/output instead of bin, but to no avail.

For now I have turned off publishing symbols, but it's making debugging hard!

Full TeamCity log can be found at:

http://dl.dropbox.com/u/2808109/Noodles_master_234.log.zip

Jul 3, 2012 at 11:03 AM

bump!

Coordinator
Jul 3, 2012 at 12:58 PM

It might be worth starting a thread with SymbolSource experts to see if that can make more sense of this, since the error is coming from their server. Their forum is https://groups.google.com/forum/?fromgroups#!forum/symbolsource

Jul 3, 2012 at 1:53 PM
Edited Jul 3, 2012 at 1:56 PM

Right - but shouldn't a PDB made with a view compiled via RazorGenerator have the correct path in it?

In the pdb*: 

c:\TeamCity\buildAgent\work\67dc14c75774fd53\helpers\BootstrapHelper.cshtml 

On disk:

c:\TeamCity\buildAgent\work\67dc14c75774fd53\Noodles.Web\helpers\BootstrapHelper.cshtml

Maybe symbolsource is doing something where it is chopping the middle out of the path, but I don't see how that makes sense. To quote pranavkm "It might be the case that the source path pragmas that we generate aren't correct and SymbolSource is trying to validate it" - could this be the case, or definitely not?

 

*I am extrapolating this from the message from SymbolSource, rather than using some kind of pdb inspection

Coordinator
Jul 3, 2012 at 3:11 PM

The line numbers in the PDB come straight from the #line pragmas we generate. Maybe these are incorrect? Symbol server aside, does debugging work?

Jul 3, 2012 at 4:17 PM
Edited Jul 3, 2012 at 4:18 PM

The generated pragma for Noodles/Noodles.Web/Helpers/NoodlesHelper.cshtml (which is in the Noodles solution, and the Noodles.Web project)

file is

   #line 2 "..\..\Helpers\NoodlesHelper.cshtml"

Are the pragmas meant to be relative to the solution directory, or the project directory? ..\..\ takes you down to the solution level...

I have just manually tested debugging, with the generated pragma (above) and with me find-and-replacing for 

#line 2 "..\Helpers\NoodlesHelper.cshtml"
and both seem to debug correctly, stepping through a View to the cshtml.

Jul 3, 2012 at 5:08 PM

I've just been testing manually find-and-replacing all the line pragmas with paths without the bits for going up a level, symbolserver seems to choke no matter what pragma I try, even if I change it so the filename exists on disk...

I'm stumped :( I don't think my problem is a problem with RazorGenerator, although I DO think it's generating the pragmas incorrectly - one level too many of ..\

Jul 4, 2012 at 12:47 PM

I believe this is indeed a problem with pragmas and the RazorGenerator magic. What SymbolSource does is take all paths as reported by the PDB file, find the shortest common relative path (so in your case I would guess (c:\TeamCity\buildAgent\work\67dc14c75774fd53 or perhaps c:\TeamCity\buildAgent\work\67dc14c75774fd53\Noodles.Web) and then try to apply that common path inside the package's src folder.

You can also use NuGet Package Explorer with our plugin to see all those PDB paths and validate the package using the same algorithm as SymbolSource implements online.

Can you post the failing package somewhere for analysis?

Jul 4, 2012 at 2:37 PM
Edited Jul 4, 2012 at 2:37 PM

I can't find your plugin - the github link is dead on your blogpost http://www.symbolsource.org/Public/Blog/View/2011-11-15/Updated_NuGet_Package_Explorer_Plugin

The file is here: https://dl.dropbox.com/u/2808109/Noodles.Web.1.1.271.nupkg

It has the standard RazorGenerator pragmas ("..\..\filename") I think

Jul 4, 2012 at 2:44 PM

Sorry for the dead link but NPE downloads plugins from a builtin NuGet feed, so it should just be there under Tools -> Plugins -> Add Feed Plugin (that's the misleading "properties" icon).

Can also post the symbols package?

Jul 4, 2012 at 3:34 PM
Edited Jul 4, 2012 at 3:38 PM

 

https://dl.dropbox.com/u/2808109/Noodles.Web.1.1.273.symbols.nupkg

 

Looking at the pragmas, they are handcoded by me to "..\filename" - if you want me to create a version with ..\..\filename pragmas (like the generator creates) let me know

Jul 5, 2012 at 9:15 AM

Ok, we know exactly what is going on. I was a bit wrong in my explanation of our path matching algorithm. Actually it first matches files only by filename, and only then resorts to relative path analysis if there are 2 files with the same name.

The problem lies in the symbol package conventions which do not cover this case. By design we only analyse the src folder. There are 2 solutions:

1. Relax our restriction to also cover cshtml in content. Are there any other types of files? Is there a VB-based Razor or anything similar that you guys are aware of?

2. Manually copy those cshtmls to src (e.g. using <files> in nuspec).

For now we'll implement a fixed as in 1. but I would like to know if this is intentional that those cshtmls are placed in content or is this an issue with nuget pack *.csproj? Do you have any nuspec <files> declarations?

 

Jul 5, 2012 at 10:21 AM
TripleEmcoder wrote:

For now we'll implement a fixed as in 1.

The fix is online and the package should upload fine now.

Jul 5, 2012 at 10:41 AM

The whole nupkg is generated that way by nuget.exe from the csproj, so it's normal usage.

Jul 5, 2012 at 11:26 AM
Edited Jul 5, 2012 at 11:26 AM

So just so we know which behaviour is correct, what should the pragmas inside \SomeSolution\SomeProject\SomeFolder\SomeFile.cshtml look like?

  1. #line 1 "SomeFile.cs"
  2. #line 1 "SomeFolder\SomeFile.cs"
  3. #line 1 "..\SomeFile.cs"
  4. #line 1 "..\..\SomeFile.cs"
  5. #line 1 "..\SomeFolder\SomeFile.cs"
  6. #line 1 "..\..\SomeFolder\SomeFile.cs" <--- what RazorGenerator currently does
  7. #line 1 "???"

 

6 does work on SymbolSource now.

Thanks for this, this is a massive help as my libraries have been ultra hard to debug!